Over the past few years, artificial intelligence has rapidly become a major force in the tech world, changing how almost every industry operates, from artistic fields to the financial sector and even management practices. LLMs (large language models) like OpenAI’s GPT and Google’s Gemini are evolving incredibly fast and have become essential tools for software engineers.

While the LLMs we have now can’t replace software engineers entirely, they serve as highly capable digital assistants. They’re particularly useful for coding tasks and debugging some routine issues. Based on my experience building AI and machine learning solutions, I’ll explain how to use LLMs to create code that can interact with external resources.

What Is a Large Language Model?

A large language model, or LLM, is a machine learning model trained on a vast amount of text data. The goal is to enable the model to understand and generate human-like text. Typically, transformers, a specific kind of neural network architecture, are used to build LLMs. These transformers use a “self-attention mechanism” that analyzes an entire input sequence at once, rather than word by word. This allows the model to understand the complete context of sentences, significantly improving its grasp of latent semantics—the real meaning and intention behind the words. This context awareness makes LLMs effective at generating human-like text.

A deeper network generally translates to a better understanding of the nuances of human language. Modern LLMs need massive amounts of data for training and may have billions of parameters, which are the values learned from the training data. This is based on the idea that greater depth leads to better performance in tasks requiring reasoning. To illustrate, training GPT-3 required a staggering 45TB of compressed text extracted from books and internet content. GPT-3 utilizes approximately 175 billion parameters to power its knowledge base.

Beyond GPT-3 and GPT-4, several other LLMs have emerged as significant players; these include Google’s PaLM 2](https://blog.google/technology/ai/google-palm-2-ai-large-language-model/) and [LLaMa 2 from Meta.

Because they’ve been trained on data that includes programming languages and software development practices, LLMs have also learned to generate code. These models can transform natural language text prompts into functional code across a range of programming languages and technology stacks. However, making the most of this powerful capability does require a certain degree of technical skill.

LLM Code Generation: Benefits and Drawbacks

While humans will likely always be needed for complex problem-solving and intricate tasks, LLMs can handle simpler coding jobs like intelligent assistants. Delegating repetitive tasks to an LLM can significantly boost productivity and reduce development time, particularly in the early stages of a project when prototyping and validating concepts. LLMs can also help with debugging by explaining code and identifying syntax errors that might be easily missed by human developers after a long coding session.

However, any code generated by an LLM should be treated as a first draft, not a finished product. It needs to be thoroughly reviewed and tested. Developers should also be aware of the inherent limitations of LLMs. Since they lack the problem-solving skills and flexibility of humans, LLMs struggle with complex business logic and challenges that demand out-of-the-box solutions. They may not be adequately equipped to handle highly specialized projects that utilize proprietary frameworks or require domain-specific knowledge. In essence, LLMs are powerful tools but still rely on human developers for guidance and oversight.

Generating Code With an LLM: Calling a Weather API

Most modern applications need to interact with external resources, which often means using APIs (application programming interfaces). An API provides a set of standardized instructions and protocols for applications to communicate with each other. Writing code to use an API often involves navigating detailed documentation that outlines naming conventions, supported operations, input/output data formats, and security requirements.

LLMs can help streamline this process. By providing the necessary information to the LLM, developers can have it generate the code for an API request. This example demonstrates how to prompt ChatGPT to generate a call to the OpenWeather API in Python, highlighting how to avoid issues like using deprecated functions. We’ll then send the same prompt through OpenAI’s API, illustrating how different parameters can affect the output of an LLM.

It’s important to note that while the OpenWeather API offers a free tier, exceeding a certain number of calls per day will incur costs.

Transforming ChatGPT Prompts into API Calls

In our example application, we want to display the current temperature and a short weather description. To see how ChatGPT can help with this, we provide it with the following prompt:

Write a Python script that uses the Python “requests” library to make a GET request to the OpenWeather API. The endpoint to use is “http://api.openweathermap.org/data/2.5/weather”. Include an API key in the query string parameters under the key “appid.” The API should provide the data in JSON format. Have the function return the current temperature in Fahrenheit.

ChatGPT then generates the following Python code:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

import requests

# Replace "YOUR_API_KEY" with your actual OpenWeather API key.
api_key = "YOUR_API_KEY"
api_url = "http://api.openweathermap.org/data/2.5/weather"

# City for which you want to get weather information.
city_name = "London"

# Parameters for the API request.
params = {
    'q': city_name,
    'appid': api_key,
    'units': 'imperial'  # Use "metric" for Celsius or "imperial" for Fahrenheit.
}

try:
    # Make the GET request.
    response = requests.get(api_url, params=params)

    # Check if the request was successful (status code 200).
    if response.status_code == 200:
        # Parse and print the JSON response.
        weather_data = response.json()
        print("Weather Data for", city_name)
        print("Temperature:", weather_data['main']['temp'],"°F")
        print("Description:", weather_data['weather'][0]["description"])
    else:
        # Print an error message if the request was not successful.
        print("Error:", response.status_code, response.text)

except Exception as e:
    print("An error occurred:", e)

Notice that we didn’t specify a location in the prompt. The generated code includes a variable city_name with a default value of “London.” Don’t forget to replace the placeholder YOUR_API_KEY with your actual OpenWeather API key.

Before testing the code, make sure you have the Python interpreter installed on your system. You can then run the code using an IDE like Visual Studio Code, or directly from the invoking the script using a command. The output for London’s weather looks like this:

1
2
3

Weather Data for London
Temperature: 53.08°F       
Description: overcast clouds

In many real-world scenarios, the data retrieved from an API needs to be stored or used by another system, such as a database or another API. This often involves converting the data into a specific format like XML, CSV, or even a SQL command to update a database table. We can ask the LLM to generate code for this custom output as well. Let’s add this sentence to our original prompt:

Format the output as a SQL command that inserts the data into a table named ‘current_weather’. The table columns are “city,” “temperature,” “weather_description,” and “update_date.”

ChatGPT responds with the same base code as before, adding a few import statements and modifying the output format:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

import requests
import json
from datetime import datetime

# …

try:
    # Make the GET request.
    response = requests.get(api_url, params=params)

    # Check if the request was successful (status code 200).
    if response.status_code == 200:
        # Parse and print the JSON response.
        weather_data = response.json()

        # Extract relevant information.
        city = city_name
        temperature = weather_data['main']['temp']
        weather_description = weather_data['weather'][0]['description']
        update_date = datetime.utcnow().strftime('%Y-%m-%d %H:%M:%S')

        # Generate SQL command for insertion.
        sql_command = f"INSERT INTO current_weather (city, temperature, weather_description, update_date) VALUES ('{city}', {temperature}, '{weather_description}', '{update_date}')"
            
        # Print the SQL command.
        print(sql_command)
    else:
        # Print an error message if the request was not successful.
        print("Error:", response.status_code, response.text)

except Exception as e:
    print("An error occurred:", e)

Executing this code produces the desired SQL command:

1

INSERT INTO current_weather (city, temperature, weather_description, update_date) VALUES ('London', 53.37, 'broken clouds', '2024-02-06 04:43:35')

However, we also get a warning about the utcnow() function being deprecated:

1

DeprecationWarning: datetime.datetime.utcnow() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.now(datetime.UTC).

To prevent ChatGPT from using outdated functions, we add this line to our prompt:

Avoid using any deprecated functions.

ChatGPT now replaces the utcnow() function with this:

1
2

# Use timezone-aware object for update_date.
update_date = datetime.now(timezone.utc).strftime('%Y-%m-%d %H:%M:%S')

The code once again generates the correct SQL command. You can test these SQL commands using various IDEs like Visual Studio Code or query editors in database management tools. In a typical web app, this SQL command would execute immediately after the API call, updating the database table in real time.

This demonstrates that with clear instructions, LLMs can structure output into virtually any format, including SQL commands, JSON, or even a call to another API.

Using the OpenAI API Instead of ChatGPT

Many LLMs offer API access, allowing developers to interact with them programmatically and integrate them directly into applications. This opens up possibilities for building custom AI assistants that can generate code for autocompletion, refactoring, and optimization. User interfaces can be tailored for specific domains and pre-populated with relevant prompt templates. Programmatic LLM integration also enables task scheduling and automation, essentially creating a virtual AI assistant that works behind the scenes.

Let’s revisit our weather retrieval task, but this time, we’ll use Python to communicate with the OpenAI API instead of manually prompting ChatGPT. Install the API using this command:

1

pip install openai

You’ll need an API key to use the OpenAI API. While there’s a cost associated with the API, new accounts typically get free access for a trial period.

After installing the OpenAI API, you can import its library into your Python code. The following code snippet sends the same prompt we used earlier, but without the instruction to output a SQL command. We’ll use the GPT-3.5 model for this example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

import openai

openai.api_key = "YOUR_API_KEY"  # Replace with your actual API key.

response = openai.chat.completions.create(
  model='gpt-3.5-turbo',
  max_tokens=1024, temperature=0.3, top_p=0.9,
  messages=[
        {'role': 'system', 'content': 'You are an intelligent AI assistant.'},
        {'role': 'user', 'content': "Write a Python script that uses the Python 'requests' library to perform a GET request to the OpenWeather API at the endpoint 'http://api.openweathermap.org/data/2.5/weather'. The API call should include an API key in the query string parameters under the key 'appid.' The data returned by the API is in JSON format. Please do not use any functions that are deprecated."},
    ],
)

response_message = response.choices[0].message.content
print(response_message )

The LLM generates this Python code, which retrieves the temperature for London:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

import requests

# API endpoint and API key.
url = "http://api.openweathermap.org/data/2.5/weather"
api_key = "YOUR_API_KEY"

# Query string parameters.
params = {
    'q': 'London,uk',  # Example city and country code.
    'appid': api_key
}

# Send GET request.
response = requests.get(url, params=params)

# Check if request was successful.
if response.status_code == 200:
    # Parse JSON response.
    data = response.json()

    # Print the temperature in Celsius.
    temperature = data['main']['temp'] - 273.15
    print(f"The temperature in London is {temperature:.2f}°C.")
else:
    print(f"Error: {response.status_code}")

Notice that we didn’t explicitly ask for the temperature in Fahrenheit this time. While the LLM doesn’t specify the units in the API call itself, it chooses to convert the temperature from Kelvins to Celsius when displaying the result.

Fine-tuning LLM Output with Parameters

When using the API, you can fine-tune various LLM parameters to influence the generated responses. Some parameters affect the randomness and creativity of the output, while others focus on repetition. While these parameters are often used for generating natural language text, they can also impact code generation.

In our previous code example, we can adjust GPT’s parameters in line 7:

1

max_tokens=1024, temperature=0.3, top_p=0.9,

Here are some key parameters you can adjust:

Let’s see how tweaking the frequency_penalty and presence_penalty parameters affects code generation.

When we set frequency_penalty to 1.5 (a higher value), the LLM adds code for exception handling, possibly to avoid repeating previous output:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

#...

try:
    # Send GET request with params and get response data in JSON format.
    response = requests.get(url, params=params)

    if response.status_code == 200:
        weather_data = response.json()

        # Print out some relevant information from the weather data.
        print("City:", weather_data['name'])
        print("Temperature:", weather_data['main']['temp'], "K")

except requests.exceptions.RequestException as e:
   # Handle any error that occurred during the HTTP request.
   print("Error:", e)

While this added error handling is beneficial, it’s important to note that the results are not always predictable. The error handling prevents the program from crashing if the external API has issues, but since we didn’t explicitly ask for it, its inclusion is more of a fortunate coincidence. Running the same code with the same parameters again might produce a different result. The most reliable way to have the LLM generate error handling code is to include those specific instructions in the initial prompt.

Setting presence_penalty to 2.0 (the highest value) has a similar effect. The LLM avoids repetition and instead places the OpenWeather API call within a function, passing the API key as an argument:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21

import requests

def get_weather(api_key):
    url = "http://api.openweathermap.org/data/2.5/weather"
    params = {
        'q': 'London,uk',  # Example city and country code.
        'appid': api_key
    }


    if response.status_code == 200:
        data = response.json()
        return data
    else:
        print("Error:", response.status_code)

# Replace "YOUR_API_KEY" with your actual API key from OpenWeather.
api_key = "YOUR_API_KEY"

weather_data = get_weather(api_key)
print(weather_data)

While putting the API call in a function is useful, passing the API key as an argument in this case is not ideal. Usually, we would pass the city as an argument. This highlights that while parameter adjustments can lead to interesting variations, they need to be carefully reviewed and potentially corrected.

Experimentation is key when adjusting LLM parameters. The results are often unpredictable, and some parameters might seem to have no effect. In general, settings that favor predictability and allow repetition tend to produce more concise and efficient code. On the other hand, settings that prioritize creativity and penalize repetition might result in more innovative solutions but with a higher risk of non-functional or irrelevant code.

Integrating LLM-Generated Code into Projects

Once you have working, optimized code from the LLM, the next step is integrating it into your project. Directly pasting the code into your existing codebase can lead to messy formatting and make future maintenance difficult. A cleaner approach is to refactor the code into reusable functions or objects. You can use the LLM for this by providing detailed instructions in the prompt, specifying arguments and desired output formats.

Let’s have the LLM create a Weather object with a get_weather function. Add this instruction to the prompt:

Place the API call within an object named “weather”. This object should have a function called “get_weather” that takes a “city” argument.

Based on this prompt, the LLM generates the following Python code:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

import requests

class Weather:
    def __init__(self, api_key):
        self.api_key = api_key
        self.temperature = None

    def get_weather(self, city):
        url = 'http://api.openweathermap.org/data/2.5/weather'
        params = {
            'q': city,
            'appid': self.api_key,
            'units': 'imperial'  # Request temperature in Fahrenheit.
        }

        response = requests.get(url, params=params)
        data = response.json()

        if response.status_code == 200:
            self.temperature = data['main']['temp']
            print(f"The temperature in {city} is {self.temperature}°F.")
        else:
            print(f"Error: {data['message']}")

# Replace "YOUR_API_KEY" with your actual API key from OpenWeather.
api_key = "YOUR_API_KEY"

weather = Weather(api_key)
weather.get_weather('London')

The LLM successfully creates a working function within a Weather object. This object can now be integrated into a larger application. However, we should still incorporate error handling to prevent unexpected crashes or potential data leaks.

Implementing Error Handling and Testing

API calls can fail for various reasons in real-world applications. To ensure our code is robust, we need to implement proper error handling. In our previous examples, the LLM included a check for a successful HTTP status code (200) and displayed an error description for other responses. However, this doesn’t cover common error scenarios like connection timeouts or DNS resolution failures.

For increased reliability, we need to incorporate error handling mechanisms, such as using a try-catch block when making the API call. We can append this instruction to our previous prompt:

For error handling, incorporate a try-catch block when making the API call. Also, ensure the code checks for an HTTP response status of 200 and displays the error description if it’s not 200.

This modified prompt results in Python code that includes a try-catch block for potential errors:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

class Weather:
    def __init__(self, api_key):
        self.api_key = api_key
        self.temperature = None

    def get_weather(self, city):
        try:
            # Construct the API URL with the provided city and API key.
            url = "http://api.openweathermap.org/data/2.5/weather"
            params = {'q': city, 
                      'appid': self.api_key,
                      'units': 'imperial'  # Get temperature in Fahrenheit.
                      }

            # Send GET request to the API.
            response = requests.get(url, params=params)

            # Check if the response status is 200 (OK).
            if response.status_code == 200:
                # Parse the JSON response
                data = response.json()

                # Extract the temperature in Fahrenheit from the response.
                self.temperature = data['main']['temp']
            else:
                # Display the error description if the response status is not 200.
                print(f'Error: {response.status_code} - {response.json()["message"]}')
        except requests.exceptions.RequestException as e:
            # Handle any request exceptions.
            print(f'Error: {e}')

While the LLM-generated code is a good starting point, we might need to make further modifications for custom functionality or specific requirements. For instance, we could log errors or set up notifications for administrators in case of configuration issues like API authentication failures or service unavailability. We could also have the code fall back to a backup API if the primary one is down.

After ensuring the code functions as intended, the next critical step is thorough testing. This involves using various test cases, including potential error conditions and edge cases. Automating these tests is recommended for faster feedback and increased reliability. To assess real-world performance, we can measure execution time, memory usage, and resource consumption to identify potential bottlenecks. Insights gained from testing and monitoring can be used to further refine our prompts and fine-tune the LLM parameters.

Looking Ahead: The Future of LLMs

While LLMs are not meant to replace human expertise, their code generation capabilities are revolutionizing software development. Not only can they speed up development cycles, but they also allow developers to quickly generate multiple code variations, choosing the most suitable one. Delegating simpler tasks to LLMs frees up developers to focus on more complex problems that require human ingenuity, such as designing innovative solutions and architecting next-generation applications. With well-crafted prompts, comprehensive testing, and API integrations, developers can leverage LLMs as powerful allies in their coding endeavors.

As more developers recognize the potential of AI, we can expect to see rapid advancements in this field. However, responsible and ethical use remains paramount. Software developers, like all users of generative AI, must be mindful of potential issues related to data privacy, intellectual property, security, unintended outputs, and biases that might be present in the training data of these models. LLMs are currently a hotbed of research and development, and with time, they are poised to evolve into seamlessly integrated, intelligent assistants that will reshape the landscape of software development.