Bitcoin maximalist.

Thoughts & Technical Writings.

How-To: Manual JSON-endpoint Testing Made Easy

| Comments

Let’s say your shiny new web application relies upon a 3rd party REST API like, say, Twilio. Those guys and gals do a really nice job adhering to REST principles when it comes to their API’s design. But as a software developer trying to communicate with their API, what are the practical implications of having a RESTful API on the other end of the wire?

Quick REST refreshment…

REST is a fairly large set of principles, but for this example we’ll focus on one aspect: The ‘R’ in REST stands for ‘REpresentational (State Transfer)’.

All URLs referenced in Twilio’s documentation have the following base:

1
https://api.twilio.com/2010-04-01

Now, we want to dig a little deeper into the “subresources” that Twilio exposes for our account. Take a look at the following URL endpoint (truncated for brevity):

1
https://api.twilio.com/2010-04-01/Accounts/AC228b9.../SMS/Messages/SM1f0e8ae6ade43cb3c0ce4525424e404f.json

Because the Twilio API is RESTful, we can observe the URI itself and garner quite a bit of information about the resource we’re requesting. In this case, it’s clearly a particular SMS instance generated by (presumably our) account ID AC228b9. The “representation” of this SMS resource is extremely intuitive, and we have REST to thank for it!

‘R’ is for ‘Representational’

But I want to focus now on something I haven’t yet mentioned regarding the URL above - specifically, the .json suffix. RESTful APIs, like Twilio’s, typically allow a client (e.g. web bowser, curl, etc.) to request a particular representation of the desired resource. JSON has become an extremely popular such representation because, “It is easy for humans to read and write… It is easy for machines to parse and generate.” Given its ease-of-use and ubiquity across the interwebs, you will inevitably run into JSON endpoints as a web developer. There are many tools for working with the JSON response, but I think I may have come across one of the best strategies particularly for REPL-driven development and prototyping enthusiasts…

Step 1: Get jq

jq is a command-line utility for parsing JSON input from STDIN (or from files, etc.; it’s a BASH utility after all). Install it with:

brew install jq

Now, play around with it - try something like this (where jq . kicks off a jq process which waits for your input from STDIN):

1
2
3
4
5
⇒  jq .
{"hello":"world"}
{
  "hello": "world"
}

Step 2: GET (via curl) your JSON endpoint

There is a great resource for working with JSON called JSON Test. We can easily combine one of the JSON Test endpoints with curl to explore the (hypothetical) JSON representation of a resource like so:

1
2
3
4
5
6
⇒  curl -s http://headers.jsontest.com
{
   "Host": "headers.jsontest.com",
   "User-Agent": "curl/7.30.0",
   "Accept": "*/*"
}

Step 3: Combine steps 1 and 2 - it’s that easy!

Now that we have jq and curl down, we simply put them together by piping curl’s STDOUT into the jq program like so:

1
2
3
4
5
6
⇒  curl -s http://headers.jsontest.com | jq .
{
  "Accept": "*/*",
  "User-Agent": "curl/7.30.0",
  "Host": "headers.jsontest.com"
}

It might not look like you get much advantage by using jq over the standard curl formatted output - but jq really shines when you want to be able to sift through a very large JSON hash. Let’s say you’re reading from a fake.json file which contains hundreds of lines of JSON (you can take a quick look at the file in this gist). That bad boy has 310 lines of JSON to be exact - ain’t nobody got time fo’ dat! We can read the file and pipe the output into our trusty little friend jq; and then we can quickly identify, say, the first person’s favorite fruit as follows:

1
2
⇒  jq ".[0].favoriteFruit" < fake.json
"strawberry"

(Note: I’m using " above because zsh passes arguments to programs a little differently than bash. If using bash, you should be able to do simply: jq .[0].favoriteFruit without the quotes.)

By doing [0] I obtain the first element of the JSON array (which contains information relating to our first person); and jq allows me to pluck the value at a given key - in this case, the favoriteFruit key.

Conclusion

By combining curl with jq, you should never again have to struggle with manually quick-checking any server-generated JSON response that comes your way. Let me know if you found this helpful in the comments below!

Comments