Turning a REST API into GraphQL Using StepZen
Brian RinaldiSo you've fallen in love with GraphQL. I don't blame you. The ability to query everything you need and only what you need is really powerful. The self-documenting nature of GraphQL and, even better, the query hinting make using it a great developer experience. Oh, and personally I definitely don't miss digging through REST API docs trying to figure out what data each endpoint provides or deal with inconsistencies between endpoints that provide similar data. The point is, there's a lot to be excited about with GraphQL.
In fact, the biggest drawback to GraphQL can be the fact that many, if not most, of the APIs developers interact with day-to-day — whether internal APIs or public APIs — are REST, leaving your newfound love of GraphQL unfulfilled. The good news is I've got a solution for you. StepZen makes it easy to convert that REST API into a GraphQL one, and even combine REST APIs (or other data sources) together into a single GraphQL backend. In this post, we walk through how to convert an existing REST API into GraphQL, connect two separate REST APIs and then query both using a single GraphQL query.
The Example
For this post, we'll rebuild the combined DEV/GitHub API that I created for my recent post about creating a developer portfolio. However, in this case, rather than rely on the StepZen CLI's stepzen import functionality to set up the schema for us, we'll build it from scratch so that you can take these concepts and apply them to your own custom REST connector using StepZen. Also, for the sake of this demo, we'll focus exclusively on creating the GraphQL schema rather than the frontend to consume it.
The end result will be a subset of the devto-github example schema in the Stepzen Schemas repository on GitHub.
Getting Set Up
First of all, you'll need a StepZen account. If you don't already have one, you can request an invite here. You'll also need the StepZen CLI to upload, deploy and test your schema on StepZen. You can install the StepZen CLI via npm.
npm install stepzen -g
Go ahead and create a project folder to work in and change directory to that folder within your command line/console. Ok, we're ready to get to work.
Planning Our GraphQL Type
The ultimate goal of this exercise is to connect the data coming out of specific API endpoints to populate data on GraphQL types. To begin with, what we want from the DEV API is the following:
- A list of articles with the article details we'd need to render these on a blog or other site.
- Ultimately, we'll need some sort of user information that can connect these blogs to a user in GitHub so that we can tie the two piees together.
The information we want is in the /articles endpoint of the DEV API. In order to construct our GraphQL type, let's look at the API documentation for this endpoint to determine what data that it returns that we'll want to use to populate our type. This will help us determine what properties we'll need on the GraphQL type (the properties don't need to be named the same between the REST result and the GraphQL type, but having a sense of what data will be there will still help us create the schema).
The DEV API returns an array of articles with the following response that contains an array of articles:
[
{
"type_of": "article",
"id": 194541,
"title": "The Title",
"description": "",
"cover_image": "image_url",
"readable_publish_date": "Oct 24",
"social_image": "image_url",
"tag_list": ["meta", "changelog", "css", "ux"],
"tags": "meta, changelog, css, ux",
"slug": "the-url-2kgk",
"path": "/stepzen/the-url-2kgk",
"url": "url",
"canonical_url": "image_url",
"comments_count": 37,
"positive_reactions_count": 12,
"public_reactions_count": 142,
"collection_id": null,
"created_at": "2019-10-24T13:41:29Z",
"edited_at": "2019-10-24T13:56:35Z",
"crossposted_at": null,
"published_at": "2019-10-24T13:52:17Z",
"last_comment_at": "2019-10-25T08:12:43Z",
"published_timestamp": "2019-10-24T13:52:17Z",
"user": {
"name": "Brian Rinaldi",
"username": "remotesynth",
"twitter_username": "remotesynth",
"github_username": "remotesynth",
"website_url": "http://stepzen.com",
"profile_image": "image_url",
"profile_image_90": "image_url"
},
"organization": {
"name": "StepZen team",
"username": "stepzen",
"slug": "remotesynth",
"profile_image": "image_url",
"profile_image_90": "image_url"
}
}
]
For most uses cases, this response probably contains way more information than you need (that's the drawback to REST after all), and it definitely does in our case. Instead, we'll take a subset of these fields:
[
{
"id": 194541,
"title": "The Title",
"description": "",
"cover_image": "image_url",
"readable_publish_date": "Oct 24",
"tags": "meta, changelog, css, ux",
"slug": "the-url-2kgk",
"path": "/stepzen/the-url-2kgk",
"url": "url",
"published_timestamp": "2019-10-24T13:52:17Z",
"user": {
"github_username": "remotesynth"
},
"organization": {
"username": "stepzen"
}
}
]
Ultimately, we'd like our GraphQL type for the article to serve getting the full article list and also a single article. To get a single article, we'll be using the published article by path endpoint (/getArticleByPath) to get the article details. This endpoint provides us with two additional fields that we'll need: body_html and body_markdown, so we'll want to accommodate those in the type.
It's also worth pointing out that the two REST endpoints have an inconsistency. In the first, tags is a comma-separated list while tag_list is an array. In the second, tags is an array while tags_list is a comma-separated list. Don't worry, we'll work that issue out.
Creating the GraphQL Type
Let's create a file for our Article GraphQL type called article.graphql. The type will accomodate all of the slimmed down list of properties that we want from the REST API above, but it flattens out the structure so that things like github_username and organization are defined at the top level. In GraphQL, those nested objects would be additional types. We'll see how to define those later, but for our purposes here, we can flatten out those properties but leave them commented out (denoted by the #).
type Article {
id: ID!
title: String!
description: String
cover_image: String
readable_publish_date: String
#tag_list: String
path: String!
slug: String!
url: String!
published_timestamp: Date!
#username: String!
#organization: String
#github_username: String!
body_html: String
body_markdown: String
}
As you can see, each property defines a data type, in most cases here this is a String. The ! on some of them indicates that they are a required property.
For this type to be useful, we also need to define a query on it. For now, let's just define a single query that returns an array of articles. You can place the query definition beneath the closing bracket of the type.
type Query {
myArticles: [Article]
}
We need to define a return type on any query. In this instance, the brackets indicate that it is an array containing instances of our Article type.
Using Our First Directive : @mock
To build out our schema for us, StepZen uses custom GraphQL directives. Directives are a special feature of GraphQL that can be used to decorate a schema or query with additional configuration. They pass this configuration to the GraphQL server, which can perform custom logic. StepZen provides a number of custom StepZen directives that tell StepZen how to create your schema and populate it with data. We'll be using many of them within this tutorial.
The first directive we'll use is @mock and it is very simple as it takes no additional configuration. It can be applied to a type to tell StepZen to populate it with mock data. Let's add it to our Article type:
type Article @mock {
...
}
That's all we need to do. Now when we query articles, StepZen will fill the result with lorem ipsum text. Let's test this out. From the command line, make sure you are in your project folder and then enter the following command:
stepzen start
Since this is the first time we are deploying the API, it will ask us to supply a name for the endpoint. This follows the format of folder-name/schema-name. This can be anything you want - the CLI will even suggest a random name for you - but let's use api/devto-github. It's a good idea to choose something that is somewhat obvious for folder-name/schema-name as this ultimately determines the URL you'll use to connect to your GraphQL API endpoint.
When the CLI finishes uploading and deploying our schema, it creates a browser window with a GraphQL query editor where we can build and test queries against our API.
The default way to test your GraphQL endpoint is from the StepZen dashboard explorer. You can get a local GraphiQL IDE by running
stepzen startwith the--dashboard=localflag.
Click the "Explorer" button at the top of the page, and it displays the myArticles query we just created. We can then add properties to a query in the editor by selecting properties in the Explorer. For example, we can query for the title and description for myArticles using the following query:
query MyQuery {
myArticles {
title
description
}
}
It returns something like the following:
{
"data": {
"myArticles": [
{
"description": "Morbi in ipsum sit amet pede facilisis laoreet",
"title": "Suspendisse potenti"
}
]
}
}
Getting mock data can be really helpful when doing some initial schema creation and testing, especially in cases where the backend may not be ready to connect to yet. But it's definitely more exciting to connect to a real backend, so let's do that next.
Getting Data from a REST API
Let's populate the myArticles query with some real data coming from the DEV API. To do this, first we need to remove the @mock from our type. Next, let's modify the query using StepZen's @rest directive. The @rest directive has a number of configuration options, but the only required one is the most important one, that being the endpoint that you want to connect to. For now, we'll just supply that.
type Query {
myArticles(username: String!): [Article]
@rest(
endpoint: "https://dev.to/api/articles?username=$username&per_page=1000"
)
}
Notice that the URL uses the username argument passed to the query to construct the endpoint URL. The value of $username in the URL is replaced by the value passed in username.
Assuming stepzen start is still running, this change is automatically picked up and deployed to StepZen. Let's update our query to match the new requirement that we pass a username (note: you may need to refresh the GraphQL query editor for it to pick up your new schema changes in the explorer whenever we change it). I'm sharing my DEV username in case you don't have one, but, if you do, feel free to replace mine with your own.
query MyQuery {
myArticles(username: "remotesynth") {
title
description
}
}
Should now return something like:
{
"data": {
"myArticles": [
{
"description": "Create a developer portfolio featuring content pulled from your DEV.to blog posts and GitHub profile and projects using Next.js and StepZen.",
"title": "Creating a Developer Portfolio using Next.js, GraphQL, DEV and GitHub"
},
{
"description": "Some tips and things to think about when choosing to run a virtual developer conference based upon my recent experiences.",
"title": "Tips for Running Your First Virtual Conference"
}
]
}
}
Nice! We barely did anything and we're already able to connect our API to real data! 💪
Adding a Configuration
At this point, it kind of doesn't make sense that our myArticles query asks for a username as an argument when it is my articles after all. Let's fix that by adding a StepZen configuration file. In this case, our configuration just enables us to set a value for username, but this is also be where you'd place connection details such as your API key if one is needed to connect to the REST API (we'll explore that later when we dig into GitHub).
In the root of our schema folder, create a file named config.yaml. Because this file can contain API keys that you do not want to publish, we highly recommend that you add it to your .gitignore. The configurations are set via YAML. You can have multiple configurations within a single config.yaml file. For now, we'll only have one:
configurationset:
- configuration:
name: dev_config
username: remotesynth
Again, be sure to replace the value of username with your own DEV username if you have one. The name of the config can be anything. This is how we'll reference this configuration within the schema. In this case, username is an arbitrary key that I want to store with this configuration.
Once we save this file, stepzen start uploads it to StepZen, however we're not putting it to use yet. Let's modify our query to remove the username argument and tell the schema to use the configuration instead.
type Query {
myArticles: [Article]
@rest(
endpoint: "https://dev.to/api/articles?username=$username&per_page=1000"
configuration: "dev_config"
)
}
The configuration property of the @rest directive allows me to reference the configuration we just created using its name. Once we save this and stepzen start redeploys it, we can query myArticles without supplying any username argument like so:
query MyQuery {
myArticles {
title
description
}
}
Mapping Data with Setters
Now that we have our query running correctly, let's deal with those nested objects and DEV API quirks that we discussed earlier (remember the properties of our article type that we commented out?). We'll do that using the setters property of the @rest directive.
By default, the value returned by a property in the JSON response of the REST API is assigned to a property of the same name in the GraphQL API. In some cases, you may want to assign that to a property with a different name. In our case, we want to reassign the values of some nested objects as well as align the value of tag_list so that it is always the comma-separated list of tags and not the array. (As a reminder, the tag_list property behaves differently in different endpoints that we'll eventually want to use in the DEV API. This would cause us issues when we implement both getting the list of articles and getting a single article from the DEV API.)
First, let's uncomment the properties for tag_list, username, github_username and organization (i.e. remove the leading #). Next, let's add the setters. The setters property takes an array of objects, each containing a field, which is the property on the GraphQL schema that we want to set, and path, which is the path to the value within the response that we want to use. For example, in the code below, the value of username on our article type is populated with the value of user.username (i.e. a nested object) within the JSON response.
type Query {
myArticles: [Article]
@rest(
endpoint: "https://dev.to/api/articles?username=$username&per_page=1000"
configuration: "dev_config"
setters: [
{ field: "username", path: "user.username" }
{ field: "github_username", path: "user.github_username" }
{ field: "tag_list", path: "tags" }
{ field: "organization", path: "organization.username" }
]
)
}
Once stepzen start uploads the updated schema, we can get the values for these properties. For instance, we can query the user's GitHub username along with the title of the article. (Note that if you are using your own DEV username and you have not connected your DEV account to GitHub, this can appear as a value of null)
query MyQuery {
myArticles {
title
github_username
}
}
Feel free to expand on this schema to implement the /article endpoint in the DEV API as shown in the finished schema, but now that we have our article type pulling from the DEV API, let's move on to connecting two different APIs.
Connecting Two REST APIs
For this example, we want to connect the DEV API to the GitHub API, so that we can get the user's bio and project information in a single query alongside their DEV articles. First, we need to implement a user type that connects to the GitHub API. Much of this will seem familiar from our experience creating the article type. Nonetheless, there are some key things to consider:
- The GitHub API requires an authorization header containing a GitHub personal access token in order to access the API. We'll need to set up a configuration to pass that token. If you don't yet have a token, you can create one here.
- Connecting two types in StepZen uses another custom directive
@materializer. We'll look at how to configure this directive.
Creating a User Type
Let's start by creating a user.graphql file in the root of our project. We'll follow a similar process as we did for the Article type, beginning by looking at the API response from GitHub and adding properties to the type to match the data we need from this API.
type User {
id: ID!
login: String!
name: String!
company: String
blog: String
location: String
email: String
bio: String
twitter_username: String
avatar_url: String
}
We haven't implemented all of the properties being sent back from the GitHub API, but these are the ones we'll need for our purposes.
Configuring the User Query
Let's add a query to User.graphql to get a user by their username. In this query, we are passing the user's GitHub login (i.e. username) and using that to construct the endpoint URL. We're also supplying the configuration value to tell StepZen what configuration to use.
type Query {
userByLogin(login: String!): User
@rest(
endpoint: "https://api.github.com/users/$login"
configuration: "github_config"
)
}
We have not yet created the configuration that is specified (i.e. github_config). Let's do that now by adding this additional configuration to config.yaml (replace out the value of MY_PERSONAL_ACCESS_TOKEN with your own GitHub personal access token).
- configuration:
name: github_config
Authorization: Bearer MY_PERSONAL_ACCESS_TOKEN
There's one last step we need to do before we can test this. We need to add user.graphql to our list of files within index.graphql.
schema @sdl(files: ["article.graphql", "user.graphql"]) {
query: Query
}
Once stepzen start uploads and deploys this to StepZen, we should be able to query our new API for a GitHub user. For example, the following query (feel free to replace my username with your own):
query MyQuery {
userByLogin(login: "remotesynth") {
name
}
}
...should return a result like:
{
"data": {
"userByLogin": {
"name": "Brian Rinaldi"
}
}
}
Nice! We're almost done here. We have two types but they are not connected yet. Let's do that next.
Connecting Types Using the @materializer Directive
StepZen provides another custom directive @materializer that essentially tells StepZen that the value for a particular field in a schema will be populated by a query on another type. We'll utilize @materializer to populate a user field on article type.
StepZen will determine what type this is connected to via the property's type, which is User. Therefore the @materializer directive only needs to specify what query it should use within that type to populate the field, userByLogin for our example. Optionally, we can instruct the directive to pass any arguments that this query requires. In this case we need to pass login, which will be equal to the value of github_username within our article type.
user: User
@materializer(
query: "userByLogin"
arguments: [{ name: "login", field: "github_username" }]
)
The complete Article type should now look like this:
type Article {
id: ID!
title: String!
description: String
cover_image: String
readable_publish_date: String
tag_list: String
path: String!
slug: String!
url: String!
published_timestamp: Date!
username: String!
organization: String
github_username: String!
body_html: String
body_markdown: String
user: User
@materializer(
query: "userByLogin"
arguments: [{ name: "login", field: "github_username" }]
)
}
Let's allow stepzen start to upload and deploy this and test it out. If we run a query like the following in our GraphiQL:
query MyQuery {
myArticles {
title
user {
name
bio
}
}
}
We sget a result similar to below (note that if you're using your own DEV account and you have not connected it to GitHub, your query won't return results or user):
{
"data": {
"myArticles": [
{
"title": "Creating a Developer Portfolio using Next.js, GraphQL, DEV and GitHub",
"user": {
"bio": "Developer advocate for StepZen. Jamstack enthusiast.",
"name": "Brian Rinaldi"
}
}
}
]
}
}
We're now getting results from both the DEV API and the GitHub API in a single GraphQL query request. That's awesome! 🙌
Where to Go From Here
We touched on a lot of things in this article:
- We learned to build a GraphQL type and queries and populate that with mock data using the
@mockdirective. - We swapped out the mock data with live data coming directly from a REST API using the
@restdirective. - We saw how to pass configuration information to supply the REST connection with authorization or other information that we need to use.
- Finally, we connected two different types pulling from two different REST APIs using the
@materializerdirective.
There's much more we could implement. For example, we haven't connected our article type to the DEV API to get a single article and its content. We also haven't created a repo type to get GitHub repositories for a user from GitHub and then connect those to the user type using @materializer. If you're curious how to do those things, I invite you to explore the finished code or, even better, pull and deploy the schema yourself using stepzen import devto-github.
