Practical coding questions answered by GPT3: is it ready for the prime time?

You can easily try GPT3 for yourself, simply register for an OpenAI account, and you will get (at the time of writing this) 18 USD of credit. Head on over to the following link:

https://platform.openai.com/playground

Here you can enter your question, and submit it. GPT3 will answer in the same text box.

There are three parameters of immediate interest:

• Model – I suggest to leave this ad text-davinci-003. You can also experiment with codex, code-davinci-002
• Temperature – 0.7 is the value I use usually (controls randomness of the response / how “creative” GPT3 will get)
• Maximum length – how many tokens should the model generate maximally? Note, the full amount of tokens you enter will count against the technical limit as well – currently 4000 tokens for text-davinci-003

Tokens, by the way, are the basic “currency” unit, you can think of them as words or parts of words. Short and common english words will usually be one token, whereas longer words will be split up into several tokens.

Using GPT3 to accelerate your coding

Every developer knows that a lot of developing includes looking for libraries, reading how to use APIs, and how to work with different objects.

I have been starting to use GPT3 as a help, while coding JavaScript, and tried it on Crystal Lang, as well.

(parts of PiCockpit are coded in Crystal Lang – because I am a fan of the Ruby language, and Crystal lang aims to have a Ruby-like syntax, and a C-like performance).

It has been useful in several occasions, e.g. to resolve questions like whether I should include spaces in the JSON syntax between keys and values. Or how to convert “require” to “import” syntax for JavaScript.

It’s good at answering questions, and it will very rarely tell you it can’t answer the question. The main thing to keep in mind, though, is that this is a statistical machine you’re dealing with.

And sometimes the answers will look plausible, but fall apart when you try the code.

JavaScript GPT3 help prompt failure example

For one project, I needed to validate e-Mail addresses using node.js. GPT3 suggested the “emailvalidator” library, and gave simple example code. When I tried to run it, it failed.

I then reviewed the documentation of the “emailvalidator” library, which indeed exists. It turns out, that this library (in my understanding) requires Redis, and a more complicated setup in general. Overkill for my simple application. GPT3 did not advise me of that fact in it’s sample code.

The similarly called “email-validator” npm library, however, does exactly what I need.

Crystal lang GPT3 query success example

crystal lang try except block give code example

begin
  # code that may raise an exception
rescue SomeError => e
  # code to handle the exception
else
  # code to be executed if no exceptions were raised
ensure
  # code always executed, whether or not exceptions were raised
end

Good to jog your memory – no need to Google, click on a page, scroll through the documentation – you’ve got the answer right at your fingertips. Saving a couple of seconds here and there will shave minutes off every day, and days and maybe even week(s) off every year of development.

Also, getting the answers in a more concise format allows you to keep your thought flow – very important while programming!

Crystal lang GPT3 help failure example 1

Crystal lang, Kemal. How can I exclude a path from before_all?

You can exclude a path from the before_all hook by using the :except option. This option takes an array of paths as an argument and will exclude those paths from the before_all hook. For example, to exclude the path '/admin/' from before_all, you would call before_all with the following code:

before_all :except => ['/admin/'] do
  # Your code here
end

Careful: the code above does not work!

Kemal is a router for Crystal Lang, you can imagine it being a similar thing to Sinatra, or to express.

The code looks somewhat plausible, I’ve tried it and it doesn’t compile. I then looked through the Kemal documentation, and the source code, and KemaL’s open issues. No mention of this mythical :except key.

GPT3 just made this up.

Here is the right way to code an exception to before_all in Kemal:

before_all do |env|
  next if env.request.path == "/admin/"
  # Your code here
end

I found this information in a kemal.cr ticket. This code is tested by me and actually compiles.

Crystal lang GPT3 help failure example 2

I also needed to convert the current Time (UTC) to a unix epoch.

crystal lang: how can I convert Time.utc to the unix timestamp?

You can use the Time#to_i method to convert a Time object to a Unix timestamp.

time = Time.utc(2020, 5, 15, 10, 10, 0)
unix_timestamp = time.to_i # 1589492200

Careful: The code above, again, does not compile!

While it could be plausible for such a method to exist, there is no such method in the Crystal Lang API for Time.

The correct code is:

unix_timestamp = Time.utc.to_unix

Conclusion: GPT3 is helpful, but test it’s output!

GPT3 indeed is helpful in solving common programming questions. It can suggest libraries, answer questions about Syntax, best practices and much more.

Treat it like an experienced programmer, who is quoting things from memory, and occasionally confuses things or makes them up. That is the nature of statistic answers – as we humans, as API developers, tend to do the “expected thing” in code (which other programmers will expect in the APIs they consume) – these answers will often work.

Also, while learning across the huge corpus of online data, GPT3 has seen A LOT of code and code examples. It has learned a lot of valid patterns.

Just like a human who is forced to quote from memory, instead of looking things up / knowing where to look things up, GPT3 will make these occasional slips.

But it will still save you time, and often point you in the right direction.

GPT3, ChatGPT and OpenAI API consulting

We offer consulting services for these AI APIs – from business uses cases, to full turnkey solutions. Get in touch today to discuss your needs and for a free initial consultation about GPT3, ChatGPT and other modern AI APIs!