GPT-4

Generative Pre-trained Transformer 4 (GPT-4) is a multimodal large language model created by OpenAI and the fourth in its GPT series. It was released on March 14, 2023, and has been made publicly available in a limited form via ChatGPT Plus, with access to its commercial API being provided via a waitlist. As a transformer, GPT-4 was pretrained to predict the next token (using both public data and "data licensed from third-party providers"), and was then fine-tuned with reinforcement learning from human and AI feedback for human alignment and policy compliance.

Observers reported the GPT-4 based version of ChatGPT to be an improvement on the previous (GPT-3.5 based) ChatGPT, with the caveat that GPT-4 retains some of the same problems. Unlike the predecessors, GPT-4 can take images as well as text as input. OpenAI has declined to reveal technical information such as the size of the GPT-4 model.

Background

OpenAI published their first paper on GPT in 2018, called "Improving Language Understanding by Generative Pre-Training." They also released GPT-1, a model based on the Transformer architecture that was trained on a large corpus of books. The next year, they introduced GPT-2, a larger model that could generate coherent text. In 2020, they introduced GPT-3, a model with 100 times the number of parameters as GPT-2, that could perform various tasks with few examples. GPT-3 was further improved into GPT-3.5, which was used to create ChatGPT.

Capabilities

OpenAI stated that GPT-4 is "more reliable, creative, and able to handle much more nuanced instructions than GPT-3.5." They produced two versions of GPT-4, with context windows of 8,192 and 32,768 tokens, a significant improvement over GPT-3.5 and GPT-3, which were limited to 4,096 and 2,049 tokens respectively. Unlike its predecessors, GPT-4 is a multimodal model: it can take images as well as text as input; this gives it the ability to describe the humor in unusual images, summarize screen-shot text, and answer exam questions that contain diagrams.

To gain further control over GPT-4, OpenAI introduced the "system message", a directive in natural language given to GPT-4 in order to specify its tone of voice and task. For example, the system message can instruct the model to "be a Shakespearean pirate", in which case it will respond in rhyming, Shakespearean prose, or request it to "always write the output of tsresponse in JSON", in which case the model will do so, adding keys and values as it sees fit to match the structure of its reply. In the examples provided by OpenAI, GPT-4 refused to deviate from its system message despite requests to do otherwise by the user during the conversation.

Aptitude on standardized tests

GPT-4 demonstrates aptitude on several standardized tests. OpenAI claims that in their own testing the model received a score of 1410 on the SAT (94th percentile), 163 on the LSAT (88th percentile), and 298 on the Uniform Bar Exam (90th percentile). In contrast, OpenAI claims that GPT-3.5 received scores for the same exams in the 82nd, 40th, and 10th percentiles respectively.

Medical knowledge

Researchers from Microsoft tested GPT-4 on medical problems and found "that GPT-4, without any specialized prompt crafting, exceeds the passing score on USMLE by over 20 points and outperforms earlier general-purpose models (GPT-3.5) as well as models specifically fine-tuned on medical knowledge ( Med-PaLM, a prompt-tuned version of Flan-PaLM 540B)".

Training

OpenAI did not release the technical details of GPT-4; the technical report explicitly refrained from specifying the model size, architecture, or hardware used during either training or inference. While the report described that the model was trained using a combination of first supervised learning on a large dataset, then reinforcement learning using both human and AI feedback, it did not provide details of the training, including the process by which the training dataset was constructed, the computing power required, or any hyperparameters such as the learning rate, epoch count, or optimizer(s) used. The report claimed that "the competitive landscape and the safety implications of large-scale models" were factors that influenced this decision.

Alignment

According to their report, OpenAI conducted internal adversarial testing on and before GPT-4's launch date with dedicated red teams composed of researchers and industry professionals to mitigate potential vulnerabilities. As part of these efforts, they granted the Alignment Research Center (ARC) early access to the models to assess power-seeking risks. As an example, ARC found that the model was able to encourage a TaskRabbit crowdsource worker to solve a CAPTCHA for it, but was not able to autonomously replicate or acquire resources. In order to properly refuse harmful prompts, outputs from GPT-4 were tweaked using the model itself as a tool. A GPT-4 classifier serving as a rule-based reward model (RBRM) would take prompts, the corresponding output from the GPT-4 policy model, and a human-written set of rules to classify the output according to the rubric. GPT-4 was then rewarded for refusing to respond to harmful prompts as classified by the RBRM.

Reception

U.S. Representatives Don Beyer and Ted Lieu confirmed to the New York Times that Sam Altman, CEO of OpenAI, visited Congress in January 2023 to demonstrate GPT-4 and its improved "security controls" compared to other AI models.

According to '' Vox'', GPT-4 "impressed observers with its markedly improved performance across reasoning, retention, and coding." ''Mashable'' agreed that GPT-4 was usually a significant improvement, but also judged that GPT-3 would occasionally give better answers in a side-by-side comparison.

Microsoft Research tested the model behind GPT-4 and concluded that "it could reasonably be viewed as an early (yet still incomplete) version of an artificial general intelligence (AGI) system".

AI safety concerns

In late March 2023, an open letter from the Future of Life Institute signed by various AI researchers and tech executives called for the pausing of all training of AIs stronger than GPT-4 for 6 months, citing AI safety concerns amid a race of progress in the field. The signatories, which included figures such as AI pioneer Yoshua Bengio, Apple co-founder Steve Wozniak, and Tesla CEO Elon Musk, expressed concern about both near-term and existential risks of AI development such as a potential AI singularity. OpenAI CEO Sam Altman did not sign the letter, arguing that OpenAI already prioritizes safety.

Criticisms

While OpenAI released both the weights of the neural network and the technical details of GPT-2, and, although not releasing the weights, did release the technical details of GPT-3, OpenAI did not reveal either the weights or the technical details of GPT-4. This decision has been criticized by other AI researchers, who argue that it hinders open research into GPT-4's biases and safety. Sasha Luccioni, a research scientist at HuggingFace, argued that the model was a "dead end" for the scientific community due to its closed nature, which prevents others from building upon GPT-4's improvements. HuggingFace co-founder Thomas Wolf argued that with GPT-4, "OpenAI is now a fully closed company with scientific communication akin to press releases for products". Like its predecessor, GPT-4 has been known to "hallucinate". The model has also been criticized for generating hateful, biased, and racist information.

Usage

ChatGPT Plus

ChatGPT Plus is a GPT-4 backed version of ChatGPT available for a 20 USD per month subscription fee (the original version is backed by GPT-3.5). OpenAI also makes GPT-4 available to a select group of applicants through their GPT-4 API waitlist; after being accepted, an additional fee of 0.03 USD per 1000 tokens in the initial text provided to the model ("prompt"), and 0.06 USD per 1000 tokens that the model generates ("completion"), is required to use the version of the model with a 8192-token context window; for the 32768-token version, those prices are doubled.

Duolingo

Duolingo integrated GPT-4 in their application through two new features, "Roleplay" and "Explain My Answer". The first version of this update is aimed only at English speakers who are learning French or Spanish, with plans to extend the features to other languages in the future.

Miðeind ehf

Icelandic start-up Miðeind ehf, which works on language preservation, was selected by OpenAI as one of six companies to participate in an early beta test program of the new model.

Khan Academy

Khan Academy uses GPT-4 to create a tutoring chatbot, which the organization names "Khanmigo". While it is in the "research phase", access to the chatbot is provided free to the students and teachers of 500 school districts who have "partnered" with Khan Academy. Public access is only offered to a limited number of users selected from a waitlist; after acceptance, a 20 USD per month fee is required to use the technology.

Be My Eyes

Be My Eyes, which helps visually impaired people to identify objects and navigate their surroundings, was the first app to incorporate GPT-4's image recognition capabilities, through a new "Virtual Volunteer" feature. The feature is an alternative to relying on human volunteers for the same tasks. The ''Be My Eyes'' "Virtual Volunteer" is in beta testing.

GitHub Copilot

GitHub Copilot announced a GPT-4 powered assistant named "Copilot X". The product provides another chat-style interface to GPT-4, allowing the programmer to receive answers to questions like "how do I vertically center a div?". A feature termed "context-aware conversations" allows the user to highlight a portion of code within Visual Studio Code and direct GPT-4 to perform actions on it, such as the writing of unit tests. Another feature allows summaries, or "code walkthroughs", to be autogenerated by GPT-4 for pull requests submitted to GitHub. Copilot X also provides terminal integration, which allows the user to ask GPT-4 to generate shell commands based on natural language requests. , while GitHub provides access to a limited number of people selected through a waitlist, the release date as well as the cost of the product are still to be announced.

Microsoft Bing

Microsoft 365 Copilot

On 17 March 2023, Microsoft announced further integration of GPT-4 into its products, revealing Microsoft 365 Copilot, "embedded in the apps millions of people use everyday: Word, Excel, PowerPoint, Outlook, Teams, and more".

Stripe

Stripe utilizes GPT-4 to help with fraud detection, and to try to improve other aspects of the user experience.

Potential applications

Multimodal AI models such as GPT-4 may offer benefits for personalized medicine (tailoring treatments and interventions to individual patients based on their unique genetic and environmental factors) as well as remote healthcare by being able to act as virtual health assistants, or by helping to identify the most effective approaches. They may also help in the areas of digital clinical trials, pandemic surveillance, and digital twin technology.

References

{{Differentiable computing
OpenAI
Large language models
2023 software