These are the news items I've curated in my monitoring of the API space that have some relevance to the API definition conversation and I wanted to include in my research. I'm using all of these links to better understand how the space is testing their APIs, going beyond just monitoring and understand the details of each request and response.08 Nov 2017
I came across the new Desktop Underwriter (DU) API from Fannie Mae which provides lenders a comprehensive credit risk assessment data that determines whether a loan meets Fannie Mae’s eligibility requirements. They have a slick new website for the project, with the tag line “building on certainty”, and a smooth HTML story to walk you through what the new DU API can do. While the API seems very exciting, and valuable, the whole production is missing one thing–the API!
I am sure you have to be a partner to get access to the API, but you can tell the whole things is being led by people who have never actually used an API. Otherwise you would give us an API to actually use, and allow us to kick the tires. A hallmark of modern APIs is that you get to play with it. Marketing materials, and a sharp single page application website isn’t enough. We need the documentation, and be able to actually see what the request and response structure is, so that we can better understand the value being generated, and how we will be integrating with it. Without this, there isn’t any value. Of course, you don’t have to make the real API 100% public, you can always create API access tiers, and even deploy a sandboxed or virtualized version of the API and data for new users, protecting your valuable resources–just do not hide the API away from us, and make us consumers beg for access.
When you hide your APIs, you leave first impressions like you did with me. Wouldn’t it be better if my first impression was all about writing a story on how cool your API was, and how all my readers should be using it? Instead, I’m using you as a case of how to not do APIs. There is no reason the Fannie Mae Desktop Underwriter (DU) API can’t be publicly available, allowing us analysts and developers to read the documentation, and kick the tires. Another thing I want to push on back on, is the use of acronyms. I had to Google what DU meant. I know this is ironic, because of API and all, but I work overtime to spell out application programming interface (API) for my (new) users on a regular basis. Please don’t assume all your API consumers will immediately know what DU is. Please help unpack your acronyms, using a glossary, or expanding them inline as you explain what you do. It will make on-boarding much easier for the edges of your target audience.
Anyways, I look forward to some day testing driving the API (someday). I looked for a good five minutes for how to onboard with the API, and eventually gave up. Maybe someone involved in the project will read this, and email me with more information about how I could test drive, so that I can do a proper write up on what Fannie Mae is up to with aPIs. I’m happy to see such large entities working to make their valuable resources available via APIs, but I’m feeling like you might not have done all your homework on what API is, but luckily I’m here as the API Evangelist to help you understand the essence of why APIs work, beyond just the technology and acronym. I look forward to helping you along in your API journey Fannie Mae.
I am a big fan of Jekyll, the static content management system (CMS). All of API Evangelist runs as hundreds of little Jekyll driven Github repositories, in a sort of microservices concert, allowing me to orchestrate my research, data, and the stories I tell across all of my projects. I recommend that API providers launch their API portals using Jekyll, whether you choose to run on Github, or anywhere else using the light-weight portable solution. I have several Jekyll templates I use to to fork and turn into new API portals, providing me with a robust toolbox for making APIs more usable.
My friend and collaborator James Higginbotham(@launchany) has launched a new minimum viable documentation (MVD) template for APIs, providing API provides with everything they need out of the gate when it comes to a presence for their API. The MVD solution provides you with a place for your getting started, workflows, code samples, reference material, with OpenAPI as the heartbeat–providing you with everything you need when it comes to API documentation. It all is an open source package available on Github, allowing any API provider to fork and quickly change the content and look and feel to match your needs. Which in my opinion, is the way ALL API documentation solutions should be. None of us should be re-inventing the wheel when it comes to our API portals, there are too many good examples out their to follow.
I know that Jekyll is intimidating for many folks. I’m currently dealing with this on several fronts, but trust me when I say that Jekyll will become one of the most important tools in your API toolbox. It takes a bit to learn the structure of Jekyll, and get over some of the quirks of learning to program using Liquid, but once you do, it will open up a whole new world for you. It is much more than just a static content management system (CMS). For me, it’s most significant strength has become as a data management system (DMS)??, with OpenAPI as the heart. I use Jekyll (and Github) for managing all my OpenAPI definitions, JSON and YAML files, and increasingly publishing my data sets in this way instead of relying on server-side technology. If you are looking for an new solution when it comes to your API portal, I recommend taking a look at what James is up to.
People often learn through example. Before I’d ever consider myself a software engineer, I’d consider myself a reverse software engineer. 93% of what I know has been extracted from the work of others. Even with 7% being of my own creation, it is always heavily influenced by the work of others. People emulate what they know, what they see, and use. This is why as an API provider you should be showcasing best practices, positive examples, and healthy blueprints of what API consumers could (should) be doing.
You can see this in action with Slack’s best practice blueprints page, where they provide six blueprints of applications that API consumers should be learning from. Slack doesn’t just provide a title, description and image of example applications, it is truly a blueprint–providing diagrams, links to documentation, code samples, and other essential knowledge you will need to successfully develop an application on Slack. Providing six solid examples that anyone can reverse engineer to understand how Slack application development could (should) work.
Slack app blueprints is just one component of a pretty sophisticated getting started section offered as part of the Slack API ecosystem. I am adding application blueprint as a building block to my getting started API research, and adding it as a dimension to my API documentation & SDK research–the overlap in these areas seem like it should be strong to me. Coming across Slack app blueprints, and writing this story has reminded me that I also need to write another piece on the Slack ecosystem, and generate an outline of all the building blocks they are using in their API ecosystem, and create an updated blueprint for successful API operations that other API providers can emulate.
The Azure Marketplace has the ability to test drive anything that is deployed in the Azure Marketplace. As someone who has to sign up for an endless number of new accounts to be able to play with APIs and API services, I’m a big fan of the concept of a test drive–not just for web applications, or backend infrastructure, but specifically for individual APIs and microservices.
From the Azure site: Test Drives are ready to go environments that allow you to experience a product for free without needing an Azure subscription. An additional benefit with a Test Drive is that it is pre-provisioned - you don’t have to download, set up or configure the product and can instead spend your time on evaluating the user experience, key features, and benefits of the product.
I like it. I want more of these. I want to be able to test drive, then deploy any API I want. I don’t want to sign up for an account, enter my credit card details, talk to sales, or signup for 30 day trial–I want to test drive. I want it to have data in it, and be pre-configured for a variety of use cases. Helping me understand what is possible.
I want all the friction between me finding an API (discovery via marketplace) understanding what an API does, test driving, then deployment of the API in any cloud I want. I think we are still a little bit off from this being as frictionless as I envision in my head, but I hope with enough nudging we will get there very soon.
I spend a lot of time trying to figure out what technology does. I read press releases, pitch emails, documentation, and marketing materials trying to get an idea of what is possible. While many of the APIs I come across are intuitive, and just make sense there is still a significant portion of them that leave me scratching my head regarding what it even does.
As developers, it can be easy to think about the SDKs you will need to support API integration with your API, but I think you are making a lot of assumptions about your consumers when you focus your initial energy here. The first step in any API integration begins with IDK and not the SDK. When a potential API consumer comes across your API, the first question to be answered is: what does this API do? If the answer is I Don’t Know (IDK), we have a problem. A problem no amount of SDKs will solve, no matter how many languages you create them in.
Every API begins with an IDK. What does the Twilio API do? What does the Stripe API do? The answer to this initial question for your API cannot be IDK! As soon as I read your press release, land on the home page of your developer area, I should know what your API does in 5 seconds or less. If I walk away from our first interaction with an IDK, the chances I’ll be coming back are pretty slim. Make sure you always address the IDK, before you get to work on your SDK, for your API(s).
I keep an eye on things that are trending daily and weekly on Github because it is a great way to discover new companies and individuals doing interesting things with APIs. While looking at this earlier this week I came across the open guide to Amazon Web Services, a pretty robust, and well organized getting started guide to everything AWS.
Here is their description of this resource out of the leading cloud computing platform:
A lot of information on AWS is already written. Most people learn AWS by reading a blog or a “getting started guide” and referring to the standard AWS references. Nonetheless, trustworthy and practical information and recommendations aren’t easy to come by. AWS’s own documentation is a great but sprawling resource few have time to read fully, and it doesn’t include anything but official facts, so omits experiences of engineers. The information in blogs or Stack Overflow is also not consistently up to date.
This guide is by and for engineers who use AWS. It aims to be a useful, living reference that consolidates links, tips, gotchas, and best practices. It arose from discussion and editing over beers by several engineers who have used AWS extensively.
I find it interesting when API providers invest in authoritative solutions like this, acknowledging the scattered and often out of date nature of blogs, QA sites, forums, and the wealth of other self-service resources available for APIs. Amazon is getting seriously organized with their approach to provider resources for developers--they have been doing this a while, and know where the pain points are.
Amazon's organized approach, the breaking down by service, and the usage of Github are all interesting things I think are worth noting as part of my research. AWS is a tough API pioneer to showcase because they have way more resources than the average API provider, but as one of the early leaders in the API space they possess some serious wisdom and practices that are worth emulating. I'll keep going through their open guide, and see what other patterns I can extract and showcase so that my readers can consider.
As I look through API portals, profiling the building blocks of successful API platform, I'm always looking for bite-size stories for my readers. I was working to complete my Instagram API definition, as part of my photo API research, and their getting started graphic caught my attention.
This three step getting started visual for Instagram is prominently available on the home page, providing a very distilled down version of what I recommend API providers usually accomplish with a dedicated getting started page.
The featured getting started area provides you with the link to register your application, and overview on authentication and authorization, and a link to all the endpoints, so you can start making requests. I am a big fan of there always being a dedicated getting started page for APIs that I am on-boarding with, but I am always a bigger fan when someone distills these building blocks into something that is way easier to understand, and execute--without all the blah blah blah.
I'm going to consider adding a feature getting started building block, in addition to having the dedicated page. When done right, I think they can accomplsh the same thing, but with one less click--reducing any friction we possibly can in the on-boarding process.
Enterprise social API Jive Software wins for the best developer landing page. Normally I tell folks, like I did with Trade.gov last week, that your landing page should have a short, precise description of what you API does—this is some seriously valuable real estate, and you have one chance to make an impression.
I would put Jive Software into the category of more making a first impression, than providing a short description of what the API does:
I wouldn’t recommend this approach for every API provider, but if you can do it right, and make a solid first impression—run with it! As an API Evangelist, and IPA Evangelist, I can seriously get behind a message like “brew up something awesome”. +1
I like how they have apps, docs, APIs, and IPAs as taps, and the taps even work when you mouseover! Seriously, all of this API shit is hard to make entertaining (trust me I know), and when you can provide this human touches while inciting innovation, they can go a long way in breaking the ice with consumers.
Right above the taps Jive Software provides the essential getting started, and register links for their APIs, which signals that beyond the fun facade, they mean business, and want to help you onboard with using their APIs. I haven’t dug deep into the Jive Software API to see what else is going on, but after this first impression it is likely I will be looking around more, now that they made a good first impression.
I’m currently reviewing the Marvel Comics API, and their approach to making their rich content available via a simple web API.
They did a great job deploying the API. To start with they published very simple and honest getting started steps:
- Sign Up - Get an API key
- Link Back - Observe the attribution and linking guidelines when displaying data from the API
- Keep In Touch - Tell us about what you're building and talk to other developers on our community page
- Build Cool Stuff!
I like their style. As a building block, getting started pages are pretty important, yet most API providers provide very dry, technical steps for on-boarding with an API—if at all.
The Marvel Comics API is a refreshing approach to explaining to developers how to get up and running with the API. Its nice to see from such a big brand.
I have a whole list of projects around open data and APIs at the Department of Veterans Affairs (VA). Additionally I have numerous other open data and API projects I'd like to tackle across other federal agencies. As I do with other areas of my work, I needed a standardized way to stabilize the datasets and APis I will need for my projects, in the same way any open data and API provider should do for their consumers.
To help support my work, and hopefully the work of others I wanted to create a baseline portal that I could use at the VA, for showing what is possible when hanging open datasets and APIs, in a full featured portal. The success of any open data and / or API portal starts with the technical building blocks, like data and APIs, but have a set of business and political building blocks that are essential to their adoption and growth.
I've spent the last three years studying the business and politics of APIs. During these three years I've looked at almost 10,000 API developer portals, and I've established a base set of what I consider the building blocks of successful API portals, with a handful in which I consider essential to success. I've always wanted a simple API portal template that would reflect this research, and my new Developer @ VA portal is the first step towards achieving this.
To start with I've included 20 of what I consider essential API building blocks:
- Landing Page
- Getting Started
- Interactive Docs
- Stack Exchange
- Direct Support
- Issue Management
- Branding Guidelines
Beyond this project running using open formats and standards, and being deployable in common cloud environments like Github, Dropbox and Amazon S3, everything is machine readable by default. The portal starts with a sitemap.json, which links to other building blocks that make up the open data and API portal, including a data.json and api.json which provide programmatic access to all resources included in the site:
- data.json - Machine readable JSON file of all data assets using common core metadata for easy access to all data stored in the portal
- api.json - Machine readable JSON, using the Swagger format describing all API resources available within the portal
Next I wanted all essential building blocks of the portal to also be machine readable by default:
- Code - JSON library of all code available for use when integrating with datasets and APIs
- Gallery - A JSON gallery of web, mobile apps and visualizations, widgets, excel and other integrations on data and APis
- Support - A JSON listing of self-service and direct support channels for the portal
- Roadmap - JSON listings of both the future with a roadmap, providing transparency into portal operations
- Changelog - A JSON listing documenting the past with all changes made to the portal
This project is the product of 3 years of research, and 3 days worth of work. So it is rough. I was able to capture my initial vision in 3 days, but still a lot of work to do. I need to let it simmer a little bit, and round off some of the rough edges and apply some polish. Then the next step is to generate a generic template from it, that will become the baseline for any other portal I derive from this work.
If there is anything you'd like to see in the future, go ahead and use any of the support channels available on the site. Once I get the base template set up, I will use that as a top level project to guide the future of this baseline for open data and API portals.
On a regular basis I review my API consumption to evaluate how I’m using various APIs, and what I’m paying for them. I depend on around 20 APIs to make API Evangelist work, and I need to make sure I’m using them to their fullest potential while also being mindful of budget.
As a part of my regular review, I am looking at the differences in pricing between three key services:
- FullContact API - I use FullContact for all my company and individual contact intelligence. I go through phases of light or heavy use depending on research projects I have going on. Full Contact provides me with per API call rates depending on the endpoint and call volume, and they limit me between four packages Trying It out (Free), Getting Started ($19/month), Gaining Traction ($99/month) and Rolling ($499/month)
- Alchemy API - I use Alchemy API to primarily pull text content from blog posts, so I can use it internally for indexing. With Alchemy I get access to three packages Free, Small Business ($250.00) and Basic ($800.00)
- AWS APIs - I use AWS for all my computer, storage, database and DNS API services. With AWS I pay by the resource, bandwidth transfer and storage and the other parts and pieces or specific actions in cloud computing. There are no packages, just modular API resources I use and get billed for.
I’m just one use case, but figured I’d share my thoughts on how I use these three API resources within my little world.
For Full Contact the jump from $19/month to $99/month isn’t too bad. I’ll lump all my processing together into a single month and get lots of work done. So I tend to toggle each month between these two tiers based upon my needs.
For Alchemy API I operate within the free tier and stick with the rate limit of 1,000 API calls per day. If a blog post doesn’t get pulled because I hit my limit, I queue it for another day when I have room within my daily limit. The jump from zero to $250.00 / month is really just too big of a jump for me to make.
My Amazon Web Services bill runs between $250.00 and $1,000.00 / month. Depending on how much traffic, harvesting, processing and other crazy stuff I’m doing. I have a buffet of compute, storage, database, IP address, monitoring,DNS and other cloud computing modules that I have developed and associated with pricing in my head, so that I can make decisions in the moment about whether I can afford to process a bunch of data, launch new API, website etc.
I have a few good friends in the API space that feel API rate limits stimulate creativity, innovation and work-arounds. I agree with that statement, and think every API has to understand its own consumer and make the decision they feel is best. But in the end, I personally like single API resource pricing based upon units, that I can scale infinitely as needed in any moment. I’m am more likely to integrate services into my world when they are independent bite size chunks, with pricing not restricted by other services or service tiers. Each module tends to have different value in my world and I like to make independent decisions about how to use just that resource, disconnected from all the other resources.
The world is starting to look different when I depend on 10-20 APIs vs 1-2 APIs, and I can imagine when I reach the point where I'm depending on 100-200 APIs I will have an even greater need to have API resources be priced independent of other services or limiting pricing tiers.
I’m doing some big data work. Ok it isn’t really, but feels like I should use that label. What I’m actually doing is analyzing the Twitter activity of top APIs.
I have the twitter handles of 100 of the popular web APIs. I have been pulling and analyzing the Tweets from each of these accounts via the Twitter API for some time now.
It is time that I scale this to about 500 API twitter accounts, and also start monitoring mentions of each of these APIs on Twitte, while also performing searches for some keywords associated with each API or a group of APIs.
At this stage of growth, it was clear I was outgrowing the regular Twitter API and since what I’m doing is more big data, than twitter client app, I am looking at the two approved Twitter resellers Gnip and Datasift.
First I went to Gnip, looked at their product and it looked like what I needed. I saw at the bottom of the page, “Get Started”. I recognize that language, clicked on it and filled out their form and I get a “Thank You, Someone Will Be In Touch”.
Ok...so I move on to Datasift. I look at their product and it too, looks like what I need. I see on the page, “Get Started”. I recognize that language, clicked on it, logged on with Twitter, added my email and password, and boom--I’m dropped into a dashboard. I am able to build streams from Twitter using different keywords, Twitter users, etc. I get freemium access, with the ability to put in my credit card and be billed for more.
Next, I got a quick email from a nice fellow at Gnip asking what I was looking to do, and if I had time in the next couple days to talk on the phone. I will definitely talk with them, but by the time I get on the phone with them I will be 2-3 days into developing my solution around Datasift. Unless Gnip has some secret pricing or some magic features Datasift doesn’t have, I probably won’t switch.
That’s the difference between self service and sales oriented web APIs. Which one are you?
One of the essential ingredients of a successful API, is a self-service area to support your API developers. In my opinion this is a no-brainer. You have to have registration, documentation, code samples, forum and other essential API building blocks available to developers in a self-service way--so they can engage with your API 24/7 without asking for access.
I’ve worked hard to make sure the CityGrid API area has a logical navigation, taking developers to the essential information they will need to learn about and integrate with the APIs:
- Getting Started
- Publisher Overview
- Ads by CityGrid Documentation
- Content by CityGrid Documentation
- Code Samples
- Usage Requirements
- Terms and Conditions
- Version History
I keep a regular flow of information going through the blog to make sure there is excellent SEO content, so developers can search for anything at Google, and the blog will support linking to whats relevant in the CityGrid Developer Center.
Even with all of this, I still get some pretty basic questions like:
- I need PHP code samples? - Immediately available on the code samples page.
- What does the API cost? - Says that it is FREE on Getting Started and FAQ.
- Can I save your data in my database? - Says it on the Usage Requirements
- Do I have to show your logo? - Says it on the Usage Requirements
That is just the tip of the iceberg. I’ve posed this question before...are there enough doers in the world to make this whole API vision work? It’s hard to tell if its just my view, while in the pit of despair, or if it is just the way it is.
It seems like there will always be some users who need hand holding, and they refuse to read the self-service material available in the API area. When I’m hacking on an API, I always make sure I search exhaustively before I ask a question, but I’m discovering a new breed of developers who won’t search, won’t ask questions--then when you reach out to them they’ll ask for what they need, no matter how basic.
It seems, that self-service API resources are essential, but there will always be a segment of our audience who aren’t quite doers, they need more hand-holding and information presented to them before they get what they need.
If you think there is a link I should have listed here feel free to tweet it at me, or submit as a Github issue. Even though I do this full time, I'm still a one person show, and I miss quite a bit, and depend on my network to help me know what is going on.