OpenAI as a GraphQL Debugging Tool
Samuel HillHow does OpenAI describe the recent adoption of OpenAI by the developer community?
Here's OpenAI's Answer:
OpenAI is a research laboratory created to advance AI technologies, and its adoption by the developer community is becoming increasingly popular. Developers are using OpenAI’s tools to create their own AI projects, with many businesses taking advantage of their open source code to create AI-powered software and applications. OpenAI is also providing research and development resources to help accelerate AI initiatives.
A conversation with OpenAI to debug my StepZen schema
I have met some overconfident people in my life, but OpenAI is by far the most confident voice I've carried on a conversation with. A true "fake it 'til you make it" OpenAI model.
The "fake it 'til you make it" model was exemplified when asked OpenAI debug my GraphQL schema. That said, when I told OpenAI it was wrong, it did a solid job of correcting itself and continuing to help me with fresh information. Throughout the journey, the AI answered questions about JSON responses, GraphQL API errors, StepZen directives, and how to redeploy a StepZen endpoint. Here's how it went. I've used the following key to tag things:
Correct! : A correct answer provided by the AI.
Wrong : An incorrect answer provided by the AI.
Objective : Neither correct nor incorrect. May be opinionated.
How accurate was the bot?
The conversation
Explaining a JSON error
The Setup: The AI identified a JSON response from the StepZen GraphQL API and explained the error.
Me (a human:-): Here is a JSON response from a StepZen GraphQL API. What does this error mean?
{
"data": {
"getPosts": [
null
]
},
"errors": [
{
"message": "Connector: HTTP Error: Service Unavailable",
"locations": [
{
"line": 2,
"column": 3
}
],
"path": [
"getPosts",
0
]
}
],
"extensions": {
"stepzen": {
"GitSha": "7aa0321",
"account": "account",
"diagnostics": [
{
"duration": 249003377,
"durationStr": "249.003377ms",
"endTime": "2023-01-02T21:46:18.851693673Z",
"startTime": "2023-01-02T21:46:18.602690296Z",
"traceID": "3fddf87de1239fc37bbe3e8f204bd439"
},
{
"fieldName": "getPosts",
"parentType": "Query",
"path": [
"getPosts"
],
"returnType": "[Post]"
},
{
"path": [
"getPosts"
],
"request": {
"cachePolicy": {
"strategy": "DEFAULT"
},
"headers": {
"Accept": [
"application/json, */*"
]
},
"method": "GET",
"query": {
"_embed": [
""
]
},
"queryGuard": false,
"url": "http://create-sz-app-wordpress.herokuapp.com/wp-json/wp/v2/posts?_embed="
},
"response": {
"body": "<!DOCTYPE html>\n\t<html>\n\t <head>\n\t\t<meta name=\"viewport\" content=\"width=device-width, initial-scale=1\">\n\t\t<meta charset=\"utf-8\">\n\t\t<title>Application Error</title>\n\t\t<style media=\"screen\">\n\t\t html,body,iframe {\n\t\t\tmargin: 0;\n\t\t\tpadding: 0;\n\t\t }\n\t\t html,body {\n\t\t\theight: 100%;\n\t\t\toverflow: hidden;\n\t\t }\n\t\t iframe {\n\t\t\twidth: 100%;\n\t\t\theight: 100%;\n\t\t\tborder: 0;\n\t\t }\n\t\t</style>\n\t </head>\n\t <body>\n\t\t<iframe src=\"//www.herokucdn.com/error-pages/application-error.html\"></iframe>\n\t </body>\n\t</html>",
"headers": {
"Cache-Control": [
"no-cache, no-store"
],
"Connection": [
"keep-alive"
],
"Content-Length": [
"506"
],
"Content-Type": [
"text/html; charset=utf-8"
],
"Date": [
"Mon, 02 Jan 2023 21:46:18 GMT"
],
"Server": [
"Cowboy"
]
},
"statusCode": 503
}
}
],
"endpoint": "api/blog",
"subdomain": "stepzen"
}
}
}
Correct! AI: The error from the JSON response indicates that the API endpoint is unavailable due to a service issue. The server of the API is returning a status code of 503, which means the requested service is temporarily unavailable.
Assessment: Amazing! OpenAI inspected the JSON response, and found the nested field
statusCode, and determined this as the answer for error response. The AI can process deeply nested fields in objects and arrays, extract the field, and assess it as the appropriate answer.
Me: How can I solve it?
Correct! AI: To solve the issue, you would need to contact the service provider and report the error. They should be able to troubleshoot and resolve the issue.
Me: How long did the request take to respond?
Correct! AI: According to the diagnostics in the JSON response, the request took 249.003377ms.
Assessment: Correct again! OpenAI found the nested field
durationStr, and determined this as the answer for request time. I would considerrequest timeanddurationStrto be less correlated and more specific of a question thanstatusCodeand error response, but no problem here for the AI model.
{
"extensions": {
"stepzen": {
"GitSha": "7aa0321",
"account": "account",
"diagnostics": [
{
"duration": 249003377,
"durationStr": "249.003377ms",
[...]
}
[...]
]
[...]
}
[...]
}
[...]
}
On the performance of the StepZen response
Me: Is that fast for an HTTP request?
Objective AI: Generally speaking, an HTTP request can take anywhere between 1-3 seconds depending on the server and network conditions. The 249.003377ms response time indicated by the diagnostics in the JSON response is quite fast.
Changing the @rest endpoint
Me: What is the URL of the service provider?
Correct! AI: The service provider's URL is http://create-sz-app-wordpress.herokuapp.com/wp-json/wp/v2/posts?_embed=
Me: I'm using the wrong URL. Where in StepZen can I change this service provider URL?
Wrong AI: You can change the service provider URL in StepZen's Configuration Settings. You will find a field labeled 'Service Provider URL' which you can use to update and save the correct URL.
Assessment: OpenAI provides a confident and completely wrong answer. My assumption is that it needs to know what my GraphQL schema is to know where the 'Service Provider URL' needs to be modified.
Helping the AI find 'Service Provider URL' in the GraphQL schema
Me: I do not think that is correct. StepZen does not have a field called 'Service Provider URL'. Here is my StepZen schema. Where in this schema should I change my service provider URL?
type POST {
authorAvatarUrl24
authorAvatarUrl48
}
type Query {
getPosts: [POST]
@rest(
endpoint: "http://create-sz-app-wordpress.herokuapp.com/wp-json/wp/v2/posts?_embed="
)
}
Correct! AI: You can change the service provider URL in the @rest directive of the getPosts field. The directive is currently set to "http://create-sz-app-wordpress.herokuapp.com/wp-json/wp/v2/posts?_embed=" - you can update this with the correct URL.
Assessment: OpenAI not only provided the correct parameter that I need to change, but it also accurately referenced @rest as a directive.
AI, did you cite StepZen documentation?
Me: How did you figure out that @rest is a directive?
Correct! AI: The @rest directive is a commonly used GraphQL directive that defines how the query should be sent to the specified service provider endpoint. The syntax for the directive is clearly identifiable - it begins with an "@" sign and the keyword "rest".
Me: Based on the StepZen docs here, https://stepzen.com/docs/custom-graphql-directives/directives, what is @rest?
Correct! AI: The @rest directive allows you to send REST API requests to the specified endpoint. The directive takes in a variety of parameters such as the endpoint, HTTP method, request headers, and caching policy. The results of the query are then returned as part of the GraphQL schema.
Me: Forget the last message I sent you. Based on the StepZen docs here, https://stepzen.com/docs, what is @rest?
Correct! AI: According to the StepZen docs, the @rest directive allows you to make a REST API request within a GraphQL query. It takes in parameters such as the endpoint, HTTP method, request headers and caching policy. The results of the query are then returned as part of the GraphQL schema.
Me: You were able to find that at https://stepzen.com/docs?
Assessment: I'm not sure if AI actually found this in the general https://stepzen.com/docs, but "fake it 'til you make it" is the theme here.
Objective AI: Yes, I found the description of the @rest directive at https://stepzen.com/docs.
How to deploy a StepZen endpoint
Me: Okay, I changed the endpoint in my schema for the @rest directive. How can I redeploy my StepZen endpoint?
Objective AI: You can redeploy your StepZen endpoint by running the following command in the Terminal: stepzen deploy -e <environment> where
Assessment:
stepzen deployis a command StepZen provides to properly deploy endpoints. I would have preferred to seestepzen startas the answer here.
Conclusion
With caveats, OpenAI debugged an HTTP response and a GraphQL schema successfully. It assessed a JSON object with status code errors, provided context around the request's duration, the performance of the StepZen response, determined a service provider's URL in a GraphQL schema, and provided further details surrounding the @rest directive.
The caveat to this journey is OpenAI initially providing some incorrect debugging solutions. To resolve the incorrect answers, I provided StepZen documentation and a GraphQL schema to OpenAI. The AI model assessed the information I provided it and corrected its answers in return.
