On June 11, 2020, OpenAI, an artificial intelligence research lab situated in San Francisco, released their latest version of their AI system that can mimic human language, GPT-3. This release is a follow up to the GPT-2 model, which was released in February of 2019. GPT-3 is a machine learning platform that is designed to generate human-like text. The system is trained on a large amount of text data, which OpenAI refers to as a “corpus”. The system then uses this corpus to generate new text that is similar to the text in the corpus. OpenAI believes that GPT-3 has the potential to be a significant advance in AI technology. The system is designed to be more efficient and accurate than previous versions of the GPT platform. Additionally, GPT-3 is designed to be more user-friendly, with a simpler interface that allows users to more easily control the system. OpenAI has released a demo of the GPT-3 system, which allows users to input a prompt and then generate text based on that prompt. The demo includes a number of different prompts, such as “What is the meaning of life?” and “Describe the color blue.” OpenAI has also released a paper that describes the GPT-3 system in detail. The paper includes results from a number of experiments that were conducted to evaluate the performance of the system.
The paragraph above was written by GPT-3(Generative Pre-trained Transformer 3) which aptly describes what GPT-3 does. Although there are some minor grammatical errors, it does seem like a genuine human-written paragraph. More surprisingly, all of the facts it has stated, such as GPT-2’s release date were all true(Go search it up). Compared to GPT-2 which was only capable of generating fake news articles, this is most certainly a huge step forward. Even more, GPT-3 was trained on a 2018 AI model meaning that it was already outdated and we could train more sophisticated AIs using a new model built from the ground up. As a result, it has created numerous headlines and taken over the AI community during the first weeks of its release.
How GPT-3 Works
To understand how GPT-3 works, we first need to understand how it is trained using machine learning. The two basic concepts of machine learning are supervised and unsupervised learning. Supervised learning is where one gives an AI large amounts of carefully labeled data that has the inputs and desired outputs and how the AI should get the desired output from an input. Unsupervised learning is like how we learn, where lots of unlabeled data is fed into the AI and it has to figure out what is what. Unsupervised learning allows the AI to be scalable and makes it suitable for more generalized tasks.
GPT-3 works by combining and improving on the skill set of its predecessors, GPT-1 and GPT-2. GPT-1 secured the training model for its successors. It used a semi-supervised learning model where the training of the AI was unsupervised while the fine-tuning is supervised. GPT-2 improved on GPT-1 by adding more datasets and parameters than GPT-1. Parameters are the values that an AI can change independently as it is learning. It also introduced new concepts such as task conditioning and zero-shot learning. Task conditioning allows GPT-2 to learn multiple tasks by using the same AI model. Zero-shot learning is a part of zero-shot task transfer and this concept allows GPT-2 to learn with no examples and it understands a task just from reading the instructions given. Finally, GPT-3 came with in-context learning along with few-shot, one-shot, and zero-shot settings. In-context learning helps GPT-3 in zero-shot transfers as it allows the AI to recognize patterns in data. As a result, when the AI is presented with new prompts or instructions, it can recognize patterns from past learning and will increase the performance of the task. Few-shot, one-shot, and zero-shot settings are also specialized cases of zero-shot task transfer. In a few-shot setting, GPT-3 is provided with a task description and it is supposed to give as many examples as possible. In a one-shot setting, there is only one example being given and a zero-shot setting has no examples at all. Of course, GPT-3 also had 175 billion parameters which are 100 times more than GPT-2. It also had more datasets and general improvements of the concepts mentioned in GPT-1 and GPT-2.
How GPT-3 is Being Used
Right now, GPT-3 has been used for all kinds of applications. Since GPT-3 is a language AI and anyone can access it. People have used GPT-3 to write articles, essays, reviews, dialogue, news reports, and creative writing. Some projects that have used GPT-3 are understanding code in Replit, turning complex legal writing into plain English, and even has been used to generate memes. However, people can also use AIs like GPT-3 for evil. For example, people can use it to generate fake reviews on sites like Amazon to either increase their product’s reputation or bring down their competition. If you instruct GPT-3 to make a positive review about something and mention some of its qualities, it will give something like this “These basketball shoes are top-of-the-line! They’re made with next-generation fabric that’s even more comfortable and durable than before. Plus, the enhanced cushioning and water-proof grip will help you stay on your feet no matter what the game throws at you.” Although GPT-3 hasn’t passed the Alan Turing test and still has some limits such as losing coherency during long sentences, most of the paragraphs make sense at least on the surface level. As AIs like GPT-3 get more intelligent, humans will have to be able to differentiate between who is a human.
How can GPT-3 be applied in the Future and What Problems it can Solve
In the future, GPT-3 can be used in the metaverse and video games where NPCs(nonplayer characters) can have meaningful conversations with players instead of following predetermined prompts. It could also be used as chatbots which will give people a better experience when asking the bot for help. With further training, it could potentially even become a teacher of sorts. As mentioned above, GPT-3 can explain complex code, summarize large text, and explain complicated concepts. For example, when I asked GPT-3 to explain what fusion energy is, it came up with this short explanation “Fusion energy is a type of energy that comes from combining atoms. It is the same type of energy that powers the sun and stars. Fusion energy is not yet available to power homes and businesses, but scientists are working on making it a reality.” Then, if you train the AI to analyze and simplify the information in a school curriculum, you will almost have a teacher.
With more training, GPT-3 could help the world solve its information overload. GPT-3 could be built into a search engine and will scan the entire internet for information every day to give several multiple perspectives answers to your question. Yet it is not without dangers, it might allow our government to have complete control over the information we consume. For example, the Russian government could simply instruct GPT-3 to spread propaganda about the Russia-Ukraine conflict. As a result, when Russians ask GPT-3 about the conflict, they will receive answers such as “The Ukrainian army has been defeated! Russia is the victor in this conflict, and the Ukrainian people will now have to answer to us. This is a great day for Russia, and a dark day for Ukraine. We will now be able to take what we want from them, and they will be powerless to stop us. All hail the great Russian army!” These messages that are generated by GPT-3 show that although AI can help our society progress, we should always proceed with caution.
