What Is GPT-J? How Can Developers Use It

If you didn't know, GPT stands for Generative Pretrained Transformers and these are AI models, mostly used for generating text. They're also called Large Language Models or LLMs. The most famous model of the GPT family is GPT-3 by OpenAI. But there're some restrictions on how you can use it and that may be a problem for some creative use-cases. Luckily, there is an open-source model named GPT-J by an organization called EleutherAI. We'll also show you where and how to use the GPT-J Playground in this post.

GPT-J vs. OpenAI's GPT-3

A difference to note between these models is the number of parameters; GPT-3 is ~170 Billion parameter model while GPT-J is only 6 Billion parameters.

We'll take a look at what GPT models, specifically GPT-J can do. And how you, as a developer, can start using them in your apps right away. Once you get a hang of it, you can use any GPT model out there.

Using GPT-J to generate text

Well, as the name suggests, these models are primarily used for generating text. So simply writing in natural language is what they are best at. If you have a product where writing text is one of the features, you should definately give GPT-J a try. Some use-cases we've seen are writing Tweets in a predefined style, extending already written paragraphs, etc.

If you're like me, you'd be wondering where you can access and start using GPT-J? Even though it's an open-source model, running it is not easy because of it's size and dependency on GPUs and TPUs. But it's also available on some model hosting providers like Grand, go ahead and try it on the GPT-J Playground.

Write a couple of words and hit 'Run', the AI will write some more for you. And that is essentially what a GPT model does; you provide it some text, and it generates some more of it. The part that is given to the model is called a prompt. A prompt is how you tell the AI how, and what to write.

Prompt Design

This could easily be a post of its own, but we'll shortly share how to design your prompts to get more than just plain text generation.

Keep in mind:

1. The AI will keep writing in the style and syntax of your prompt.
2. If we do not provide any example in the prompt, it's called single-shot prediction. If we do, it's called few-shot prediction.
3. GPT-J produces a lot better results with few-shot.

Example prompt

You can use the prompts to do a lot of things. For an example, let's design a prompt that'll make the AI write SQL queries for us.

Generate SQL query from text
input: select all the users from table 'users'
output: SELECT * FROM users
input: show me all employees who are older than 38 years in age
output: SELECT * FROM employees WHERE age > 38
input: get name and address of the students enrolled in the science class
output:

A few things to note from the above prompt:

1. First line instructs the AI what we're trying to achieve
2. The same pattern of input and output to make AI repeat it too
3. We left the prompt at 'Output:' and that's where the AI will start writing from.
4. We used the character sequence of '<newline>' to tell Grand to stop the generation when a new line (\n) character is found.
5. Different wording was used in input field to not make the AI biased towards same kind of text.

Copy the above prompt into Grand's Playground, set the temperature to the lowest (0.1) and press 'Run', you'll see that it'll write the SQL query of the text and then stop. For me, it generated: SELECT name, address FROM students WHERE class = 'science'. Which is pretty good! 🪄🎩

Don't worry about the other parameters like Top P, Top K, etc. for now. We'll do a separate post to explain them.

With API

If you want to use the above prompt using the API, all you have to do is call the /generate endpoint, and you'll get the generated text in the response.

Request:

{
 "text": "Generate SQL query from text\n\ninput: select all the users from table 'users'\noutput: SELECT * FROM users\ninput: show me all employees who are older than 38 years in age\noutput: SELECT * FROM employees WHERE age > 38\ninput: get name and address of the students enrolled in the science class\noutput:",
 "max": 40,
 "creativity": 1,
 "stop": "<newline>",
 "only_response": true,
 "model_id": "Ukg1ClHdlG03aG7cAu8J",
}

Response:

{
 "ok": true,
 "id": "193b72d7",
 "data": {
  "text": " SELECT name, address FROM students WHERE class ='science'\n"
 }
}

Notice how the API returned a space before the text and a new line after it. You'll just have to strip that out.

🛑 Note: Be careful when using the output of AI-generated content. I don't need to tell you what would happen if it generated DELETE, instead of SELECT * 😱

Convert the prompt into a Playbook

You've seen how we used the entire prompt when calling the API. If we wanted to generate a different query/output, we'd have to change the input: block. What if you could just save that prompt, and call the API with its ID and input text only? Well, you can, with Playbooks.

A Playbook is simply a saved, structured and reusable prompt.

Let's convert that prompt above to a Playbook. All we have to do is create a new Playbook and do some copy-pasting. Add instruction, input and output and then the examples.

That's all, give it a try now, and it should create an SQL query for you.

Now that I have the prompt converted into a Playbook, I can use the /playbook/use API to get SQL query just by passing its ID and the input text.

Request:

{
 "playbook_id": "lKoXzCyPImEtItllr4yf",
 "input": "get name and address of the students enrolled in the science class"
}

Response

{
 "ok": true,
 "id": "2b8b88e3",
 "data": {
  "text": "SELECT name, address FROM students WHERE class ='science'"
 }
}

Looks much cleaner and easier, doesn't it?

Bonus: You can change the 'Who can access' option from Settings to allow others to use your Playbook too. Here's the Playbook I made for SQL generation if you wanna try.

Other Creative Uses of GPT-J

Translation, text summary, rephrase text and even - wait for it - code generation! Yes, you can generate code using GPT-J. Don't believe me? I've made a Playbook to generate JavaScript function from a text, try it for yourself. OpenAI has a pretty good collection of what GPT models are capable of. You can try those prompts with GPT-J too, but you'll have to provide a few more examples (few-shot) for them to work.

Using GPT-J Through API

Having API access is what matters most to us developers. In the case of GPT-J, Grand provides API access not only through a well-documented REST protocol but also through WebSockets. If you have a product which could use real-time text generation, use WebSockets API to wow your users.

Some Ideas on What to Generate With GPT-J

If you're looking for a challenge or just fun ideas to try with GPT-J, here are some:

• Generate RegEx patterns
• A conversational chatbot
• A color palette generator given a theme name
• Find grammar mistakes in the text
• Username generator
• Generate meta description from a paragraph
• Extract names and entities from text into a JSON object

If you create a product using Grand please reach out to us, and we'd love to share it with the community.