IAS Gyan

Daily News Analysis

Microsoft gets exclusive license to use GPT-3 language model. What does the model mean?

7th October, 2020 Science and Technology

Context: Microsoft recently received an exclusive license to use OpenAI’s GPT-3 (Generative Pre-trained Transformer) language model in its own products and services.

  • The model uses deep learning method to create human-like text in real-time.
  • The third-generation model built by San Francisco-based AI research company OpenAI is available to developers via the OpenAI application programming interface (API), which can be used to develop applications and services.

GPT-3 and its applications

  • GPT-3 has 175 billion machine learning (ML) parameters, over 100 times more than its predecessor.
  • The ML parameters are crucial for solving problem as they look into multiple configurations to approach a specific task.
  • It also represents the skill and complexity level of the model.
  • Advantage quantum system has more than 5000 qubits, and 15-way qubit connectivity.
  • Its expanded hybrid solver service can run problems with up to one million variables.
  • D-Wave commercialises its first 5000+ qubit quantum computer
  • The latest generation has been designed to understand inputs in English and to generate output with minimal interactions or adjustments from a user.
  • GPT-3 can write (including long-form generative text), translate, comprehend text, and answer closed book questions, reason common tasks, and code.

IBM plans to build an 1121 qubit system. What does this technology mean?

  • GPT-3 is trained on Microsoft cloud’s AI supercomputer with various datasets, which consist of text either posted or uploaded on the internet.
  • The internet data includes a version of the Common Crawl dataset, an expanded version of the WebText dataset, two internet-based book databases, and English-language Wikipedia.
  • For the model’s contextual learning, it has been trained to predict answer with only a description of the task, without any examples, with just one example of the task, or by providing few examples.

Limitations and possibilities

  • According to OpenAI, GPT-3 has the tendency to express incorrect text information confidently, and it can provide reasonable output when given inputs are similar to the ones present in its training data.
  • The training data mostly has texts in English, meaning the model is best suited for classifying, searching, summarising, or generating in that language.
  • The API can be integrated into a particular product, or used to develop an entirely new application.
  • As it provides a general-purpose “text in, text out” interface, users could try it on virtually any English language task.
  • OpenAI also expects the model to be used by researchers to better understand the behaviours, capabilities, biases, and constraints of large-scale language models.