CFCamp Notes -- API Management from the Trenches, Matt Gifford
October 25, 2015
APIs are where it's at .
We should all be building APIs for most things these days.
DISCOVERY --
is the API still active?
how strong is the community?
what kind of feedback is there on it?
try not to rely on ONE api to manage everything under your code
look at them as micro-services
pick multiple APIs to deal with manageable things.
micro-services are where it's at
ex: hotel booking
1 api for hotel rooms
1 for the weather on those days
1 for travel, 1 for car booking, etc.
1 hotel site, but broken down into sections - micro services
abstraction --
consider this from the start
weather API. if it's used in your site…what if the Weather API gets canceled.
if you deal specifically with the (for ex) Yahoo site API and it gets canceled, it's difficult to swap that out of your code. write an abstraction layer so you can swap APIs in/out easily.
Version Monitoring --
Twitter API changed their authentication and "broke" the API (it was public knowledge but still a challenge) when they moved it to OAuth
API Changelog (via Digital Ocean)
if you're using a public API, look on here first
if it's there, you log in, find the API, click "follow"
if that api changes, it will notify you so you're never caught by surprise
will notify you AS the change happens. they "poll" the public APIs. as SOON as the change happens, your'e notified, so you can react very quickly.
Network --
connecting to the API
if it's a private API, we might be connecting to the client, dealing with firewalls, etc.
traceroute
Fiddler
Authentication --
really hard
Twitter's OAuth connection, for example
DONT rewrite authentication if you don't have to. it's REALLY hard. use open source libs if you can so you have fewer headaches.
Test in Isolation --
first things first -- interact with the API directly
isolate all the little bits as you go
there are OAuth token generators that will generate the tokens/secrets for you. use these as part of your testing process.
Direct API calls
take the wrapper out and just hit the API directly for testing. so you know your wrapper isn't causing a problem.
can increase speed, interact much quicker with the API. instant gratification.
Scope --
check the moving parts
REST Client for Firefox plugin
any time you need the various HTTP methods, this client plugin can make testing much easier
cURL -- command line easy way to test it as well.
curl -i -X PUT [url goes here]
HTTPie
same as cURL but much simpler command line syntax
POSTMAN
amazing.
free and premium versions
the new version 2 is a big improvement
Payload --
Mock API --
to make sure what we're building is right
Schema
discuss as early as possible
what things will we pass in/out as parameters? how will it look?
the earlier we agree, the sooner people can start building against the API and save time.
Rate limits
another benefit of a mock API, rate limits are easier to deal with.
have your mock API run locally so you can work anywhere, don't need a wifi connection, etc.
mockable.io
apiary.io
mocky.io (similar UI to POSTMAN)
a fake API might be better than the online resources like mocky.io -- if your wifi goes down, the local fake API will continue working.
also means less security, noone trying to hit Mocky.io, etc.
everything is local, everything is fake. we can drastically minimize dataloss.
fake APIs can also be dynamic
can create fake data, etc
WebPro Dyson
various other Node-based options
json-server
another good one for local fake APIs
proxying
tunneling
various proxy tools allow us to do this
if someone else needs access to your local fake API
can expose a port to make that fake api visible to someone else
charlesproxy.com
httpkit.com/wiretap
runscope.com/docs/inspector
ngrok.com
Cache Management --
always make sure your data is clean and fresh
if your' working locally, don't worry about about caching. you won't gain enough performance locally to worry about it.
author debugging--
FIRST things first -- make sure your API actually WORKS
Logging --
as an author, logging is a top priority
consider logging as early as possible. NOT an afterthought
will need to see what's going on in your API, regardless of it it's public or private.
Find Issues --
add identifiers to URLs so you can track a particular request all the way down the stack
allows you to do a lot of analysis, see what's happening at a granular level
Cloud Logging
don't need to dump to massive data files any more. don't need mysql dumps any more
sumologic.com
-- real time searching logging facility, dashboards, monitoring, etc.
logstash.net
part of the Elastic family
can parse logs and pipe them to other services, etc.
loggly.com
nice dashboard and search capabilities, etc.
Logging is cruicial
not just error logs, but can also log exception logs
but if it's all in the same place, it's harder to analyze -- massive log with all these exceptions + everything else in there
better - segment the 2, don't send exception logs and trace logs to the same place
bugsnag.com
java wrapper for if you're doing CFML stuff
airbrake.io
similar to bugsnag
getsentry.com
similar to AirBrake and Bugsnag
pricing levels on the tools can vary
Monitoring --
we need a central area to inspect everything.
pipe all that info to a central dashboard
can build your own, or use cloud services
zapier.com
apimetrics.io
runscope.com/radar
-- can run unit tests similar to Selenium but more specific to APIs
ducksboard.com
-- great dashboard, can get everything thrown into 1 place
Load Testing
if you are launching something, get ti right. you don't want products to fail
you'll have al to of people hitting it
don't rush your API out the door without testing it
make sure a lot of people can hit it at once
github.com/JoeDog/siege
- can see how many requests/sec you can serve
loader.io
depends on an network connection to work
github.com/wg/wrk
great benchmarking tool
>> cmd -t12 -c400 -d30s http://site:8080/testthis.html
running 30s test @ site
12 threads
400 connections
super easy to use
Other considerations for authors -
as an author you have a lot of people depending on you
we don't want those people to be frustrated
first -- DOCUMENTATION!
let ppl know how to acutely use it
if you want it to be successfully adopted, build a rich suite of docs for your API
what can users send thru, what responses come back
basically spoon-feed them
they'll thank you for it if you help them out
various tools to help write docs
apibluepring.org
-based on markdown files, generates docs for you
raml.org
swagger.io
- nice tool, available in lots of languages.
- uses YAML
write .yaml file
will generate a local faked api for you
CF12 can import Swagger into the API manager in CF
bit.ly/designApi
-- great blog post on the benefits of designing your API using these tools
SDKs --
provide an SDK if possible, a CF wrapper, whatever
it might not be perfect, but provide the helping hand whenever possible.
authors -- as part of the help, allow users to talk to you
users -- ask others don't just work by yourself in a box. look for assistance.
follow these people on Twitter for api stuff
@kinlane
@johnsheehan
@synedra
above all else, have fun with it.
if you'er building something, there are lots of things to consider.
building an API isn't necessarily easy
"i can just hit an end point, i can just hit a GET URL"
it's not quite as easy as that. have to consider the RIGHT way of returning that data.
do you auto-redirect the API?
do you give them a status code? which one? is it 200 for everything?
there are specific rules, but you have to dig deeper to find them.
Book: REST Assured by Adam Tuttle
restassuredbook.com