post type:


GPT-4 image input – can you use photos with ChatGPT?

Amaar Chowdhury Updated on by

Now that OpenAI have launched its multimodal language model, you might be interested in what GPT-4 image input is capable of.

GPT-4 introduced multimodal models to ChatGPT, and one of the theorized new forms of input is images. Before, ChatGPT could only be trained with textual input, however, advancements in technology have reared it for a total change in paradigm.

Does this mean ChatGPT can have images input to it? ChatGPT Plus, at the time of writing, doesn’t allow you to do this. It’s only possible with the GPT-4 API, which you can only access as a developer.

EXCLUSIVE DEAL 10,000 free bonus credits

Jasper AI

On-brand AI content wherever you create. 100,000+ customers creating real content with Jasper. One AI tool, all the best models.

Copy AI

Experience the full power of an AI content generator that delivers premium results in seconds. 8 million users enjoy writing blogs 10x faster, effortlessly creating higher converting social media posts or writing more engaging emails. Sign up for a free trial.
ONLY $0.01 PER 100 WORDS

Originality AI detector

Originality.AI Is The Most Accurate AI Detection.Across a testing data set of 1200 data samples it achieved an accuracy of 96% while its closest competitor achieved only 35%. Useful Chrome extension. Detects across emails, Google Docs, and websites.
*Prices are subject to change. VideoGamer.com is reader-supported. When you buy through links on our site, we may earn an affiliate commission. Learn more

What can you do with GPT-4 image input?

GPT-4 image input allows you to receive natural language, code, instructions, or artificial opinions as a response to a photo.

This means that you’re going to be able to input a unique image, alongside a set of clear instructions, questions, or opinions, and GPT-4 can return a structured answer that uses both sets of data as inputs. For example, you might enter an image of a pattern of shapes, and ask GPT-4 which shape completes the pattern, though of course there are more complex and creative usages possible with the new update.

A better example of this could be sets of graphs or data, and you could extrapolate advanced business strategies based on this information. Uses like this, of course, might also be helped by the new addition of ChatGPT plugins, and the recent implementation of ChatGPT web browsing.

Examples of GPT-4 image processing

This image displays GPT-4's image processing abilitiies.

It shows a meme from reddit, and the AI responds with how it is funny.

This is an example of what GPT-4’s image input and processing can do, via OpenAI’s GPT-4 whitepaper. It takes a popular Reddit post and explains what is funny about it.

Being able to process the image and determining what’s funny about it is honestly a fantastic display of the current advancements in modern technology – and they go further and explain the ‘steerability’ of the new GPT-4 model.

What is steerability?

Steerability is effectively the user’s ability to modify and manipulate the ‘personality’ of the AI. Now, you can proscribe the AI with its own custom behavior, which OpenAI is referring to as a ‘jailbreak.’ You can therefore use steerability to pull natural language responses from GPT that are adhered to what you particularly want.

Can you input images to ChatGPT?

At the time of writing, ChatGPT doesn’t allow you to input images through the user interface. This is likely due to the primitive nature of GPT-4, which is still being trained and developed. The only way for you to attempt image entry to GPT-4 is through the developer API.

Following on from the release of ChatGPT web browsing – we imagined it would be possible to ask ChatGPT to describe an image after providing it with a link. However, unfortunately, it was only able to provide a description of the image that was likely based on the image’s file-name.

Here’s the result of when we tested this out.

Can GPT-4 generate images?

GPT-4 is strictly a multimodal language model. While it can receive varying forms of data input, it can still only return natural language responses. If you’re seeking artificial intelligence which can return images as a response, we might suggest Midjourney.

Will ChatGPT generate images?

We can’t speak for OpenAI, however they already have their DALL-E image generation API. While it differs from GPT, and what ChatGPT’s purpose is (the clue is in the name, chat), it’s possible that one day we see ChatGPT integrate both the GPT and DALL-E models into one user interface.

However, there doesn’t seem to be much point to this, unless you want an artificial friend you could send unlimited memes to, who has no choice but to reply to you.

In any case, ChatGPT currently doesn’t have the ability to generate images, nor have OpenAI mentioned anything suggesting this could happen. At the end of the day, Artificial Intelligence is better suited to being laser-focused on one purpose, and creating a chimeric robot of loads of features stapled together might not be the optimal outcome.

Can ChatGPT interpret images?

With GPT-4, ChatGPT has the ability to analyze and interpret images. To be clear, these include photographs, diagrams and screenshots.

To give you an example of this in your day-to-day life, you could enlist the help of ChatGPT to help you plan a leftovers meal. By simply taking a photo of items that you have in your cupboard, ChatGPT can give you recipe recommendations based on these images. Pretty impressive, right?

Can ChatGPT make art?

ChatGPT can make art, though you’ll need a bit of your own creativity for the most interesting results. It’s worth noting that ChatGPT is not an AI art generator that produces stylistic results like Midjourney. However, you can use ChatGPT to generate the text input for Midjourney.

Speaking of text generation, ChatGPT can generate poetry, though these results lean more towards the amusing than the profound or beautiful.

Can ChatGPT create ASCII art?

ASCII art is a popular type of digital art work made from ASCII characters that can be shared easily between online platforms. ChatGPT can create ASCII art, though it will struggle with some of your requests. While it coped fine with creating ASCII art of a cat, we didn’t think it did particularly well with out request of creating ASCII art of the lesser known capybara.

Frequently Asked Questions

Is ChatGPT an image generation AI?

ChatGPT does not make images, but GPT-4 takes them as an input via the API.