Now that OpenAI have launched its multimodal language model, you might be interested in what GPT-4 image input is capable of.
GPT-4 introduced multimodal models to ChatGPT, and one of the theorized new forms of input is images. Before, ChatGPT could only be trained with textual input, however, advancements in technology have reared it for a total change in paradigm.
Does this mean ChatGPT can have images input to it? ChatGPT Plus, at the time of writing, doesn’t allow you to do this. It’s only possible with the GPT-4 API, which you can only access as a developer.
Jasper AI
Copy AI
Originality AI detector
What can you do with GPT-4 image input?
GPT-4 image input allows you to receive natural language, code, instructions, or artificial opinions as a response to a photo.
This means that you’re going to be able to input a unique image, alongside a set of clear instructions, questions, or opinions, and GPT-4 can return a structured answer that uses both sets of data as inputs. For example, you might enter an image of a pattern of shapes, and ask GPT-4 which shape completes the pattern, though of course there are more complex and creative usages possible with the new update.
A better example of this could be sets of graphs or data, and you could extrapolate advanced business strategies based on this information. Uses like this, of course, might also be helped by the new addition of ChatGPT plugins, and the recent implementation of ChatGPT web browsing.
Examples of GPT-4 image processing
This is an example of what GPT-4’s image input and processing can do, via OpenAI’s GPT-4 whitepaper. It takes a popular Reddit post and explains what is funny about it.
Being able to process the image and determining what’s funny about it is honestly a fantastic display of the current advancements in modern technology – and they go further and explain the ‘steerability’ of the new GPT-4 model.
What is steerability?
Steerability is effectively the user’s ability to modify and manipulate the ‘personality’ of the AI. Now, you can proscribe the AI with its own custom behavior, which OpenAI is referring to as a ‘jailbreak.’ You can therefore use steerability to pull natural language responses from GPT that are adhered to what you particularly want.
Can you input images to ChatGPT?
At the time of writing, ChatGPT doesn’t allow you to input images through the user interface. This is likely due to the primitive nature of GPT-4, which is still being trained and developed. The only way for you to attempt image entry to GPT-4 is through the developer API.
Following on from the release of ChatGPT web browsing – we imagined it would be possible to ask ChatGPT to describe an image after providing it with a link. However, unfortunately, it was only able to provide a description of the image that was likely based on the image’s file-name.
Here’s the result of when we tested this out.
Can GPT-4 generate images?
GPT-4 is strictly a multimodal language model. While it can receive varying forms of data input, it can still only return natural language responses. If you’re seeking artificial intelligence which can return images as a response, we might suggest Midjourney.
Will ChatGPT generate images?
We can’t speak for OpenAI, however they already have their DALL-E image generation API. While it differs from GPT, and what ChatGPT’s purpose is (the clue is in the name, chat), it’s possible that one day we see ChatGPT integrate both the GPT and DALL-E models into one user interface.
However, there doesn’t seem to be much point to this, unless you want an artificial friend you could send unlimited memes to, who has no choice but to reply to you.
In any case, ChatGPT currently doesn’t have the ability to generate images, nor have OpenAI mentioned anything suggesting this could happen. At the end of the day, Artificial Intelligence is better suited to being laser-focused on one purpose, and creating a chimeric robot of loads of features stapled together might not be the optimal outcome.
Can ChatGPT interpret images?
With GPT-4, ChatGPT has the ability to analyze and interpret images. To be clear, these include photographs, diagrams and screenshots.
To give you an example of this in your day-to-day life, you could enlist the help of ChatGPT to help you plan a leftovers meal. By simply taking a photo of items that you have in your cupboard, ChatGPT can give you recipe recommendations based on these images. Pretty impressive, right?
Can ChatGPT make art?
ChatGPT can make art, though you’ll need a bit of your own creativity for the most interesting results. It’s worth noting that ChatGPT is not an AI art generator that produces stylistic results like Midjourney. However, you can use ChatGPT to generate the text input for Midjourney.
Speaking of text generation, ChatGPT can generate poetry, though these results lean more towards the amusing than the profound or beautiful.
Can ChatGPT create ASCII art?
ASCII art is a popular type of digital art work made from ASCII characters that can be shared easily between online platforms. ChatGPT can create ASCII art, though it will struggle with some of your requests. While it coped fine with creating ASCII art of a cat, we didn’t think it did particularly well with out request of creating ASCII art of the lesser known capybara.
Frequently Asked Questions
Is ChatGPT an image generation AI?
ChatGPT does not make images, but GPT-4 takes them as an input via the API.
