Dr. Chen Zhaohui, a "Dual Innovation Talent" of Jiangsu Province and Chief Scientist of Manbang Group, is responsible for Manbang’s big data platform and data innovation business. Before joining Manbang, he worked as a Big Data and AI Researcher at Alibaba Cloud, where he led the R&D and delivery of the Urban Brain data resource platform and algorithms. Prior to Alibaba Cloud, he was a co-founder of Celential.ai, a Silicon Valley startup. Dr. Chen graduated from Xi'an Jiaotong University and obtained his PhD from the John A. Paulson School of Engineering and Applied Sciences at Harvard University. He has long held positions in R&D and technical management at well-known Internet companies in Silicon Valley, including Oracle, Yahoo, and eBay.
The development and successful application of artificial intelligence (AI) technology have become the most significant new phenomenon in the field of science and technology in the 21st century. From the current progress, a scientific understanding of AI principles has exceeded the scope of the existing scientific system. Obviously, AI is a product of the development of human science and technology, and AI science will also be an inevitable goal to be achieved in the progress and development of human science.
Large language models are deep neural networks with hundreds of billions or more model parameters. They use autoregressive learning methods to train on massive unlabeled text and image data. Since 2018, companies and research institutions such as Google, OpenAI, Meta, Baidu, and Huawei have successively released various models including BERT and ChatGPT, which have performed excellently in almost all natural language processing tasks. In 2019, large models experienced explosive growth. In November 2022, ChatGPT 3.5 was released, allowing users to interact with the system using natural language to perform various tasks such as question answering, classification, summarization, translation, and chatting.
With the popularity of ChatGPT, the AI field has ushered in a new wave of investment. Large models have demonstrated strong ability to master and understand human natural language and world knowledge. Through the enhancement of multi-modal data, they also possess the ability to understand and reason about graphic, image, and audio data—this has aroused widespread concern about the technology and application of large models. Nanjing Innovation Investment Group has invited Dr. Chen Zhaohui, Chief Scientist of Manbang Group, to interpret with us the impact of pre-trained large models in various industries.
Nanjing Innovation Investment Group
Pre-trained large models remain popular. Why has such a hotspot formed in the AI field?
Chen Zhaohui
Knowledge question-answering systems based on information retrieval have been attempted for a long time. Around the same period as Google, there was a startup called PowerSet in Silicon Valley that summarized answers directly from various web contents and fed them back to users. Google also launched its own Knowledge Search service based on a similar concept, integrating it into its web search results page. At the beginning of this century, the explosive growth of computing power and storage promoted a significant increase in the scale of neural networks in machine learning, and various models based on deep learning achieved important breakthroughs. OpenAI has gone through a relatively long R&D process, with continuous investment from ChatGPT 1.0 onwards, accumulating what was once considered an enormous amount of computing resources, until the recent launch of the latest ChatGPT 4.0 Turbo.
The latest large models have demonstrated the ability to deeply understand human conversations. In addition to knowledge integration, they also possess a high level of reasoning logic. In specific fields, through learning from massive data, models have developed understanding and reasoning capabilities that surpass ordinary humans. The OpenAI team itself has also stated that it cannot fully understand the "intelligence" emerging from ChatGPT. It can be relatively certain that most industries will face reshaping and transformation driven by large models.
Nanjing Innovation Investment Group
What are the similarities and differences between large models and traditional machine learning?
Chen Zhaohui
Early research in machine learning focused heavily on the extraction and selection of feature dimensions, including natural attribute features, statistical features, and various features (or labels) subjectively designed by humans based on business scenarios. Deep learning enables automated feature extraction, but it still relies on a complex optimization process, especially the selection of training datasets.
In contrast, pre-trained large models capture statistical patterns in corpora by "feeding" a large amount of general data. Traditional machine learning methods usually require the reconstruction of a complete training dataset when facing new problems. Different from this, due to the support of massive pre-training in advance, pre-trained large models usually only need relatively small amounts of data and prompting training in various vertical fields to achieve good performance.
Deep learning has provided a huge impetus for the research of large models. The capabilities currently demonstrated by large models mainly benefit from the promotion of the following key factors: 1) Large-scale, multi-modal training datasets; 2) Almost unlimited super computing power (with high costs); 3) More advanced learning frameworks—for example, the Transformer architecture is more effective at extracting contextual attention than the previous CNN (Convolutional Neural Network) architecture.
Nanjing Innovation Investment Group
What are the profit models based on large models?
Chen Zhaohui
The commercial monetization of large models is very important. Compared with the huge investment costs, it is still unknown whether the current revenue can cover various cost expenditures. At present, the main known methods for commercial implementation are as follows:
Basic services based on general large models: Algorithm service calls can be sold as standardized products, forming service providers similar to cloud computing resources. In this area, various large models will show advantages in comparisons of vendor services—overseas companies such as OpenAI and domestic ones such as Zhipu AI have performed well.
Vertical large model applications in specific fields: By adapting to industry/industrial knowledge and problems, specialized services in specific fields are provided. For example, Stability AI launched text-to-image and image-to-image services. The company has obvious technical advantages in controlling the quality of output images, thus attracting a group of users.
In-house large model applications in enterprises across various segments: Enterprises in various niche industries will also build large model applications in their respective tracks based on various open-source pre-trained large models to help the industry reduce labor costs and communication costs.
Currently, the value of large models is mainly reflected in the "cost reduction" aspect. Whether large models can drive the emergence of new business demands is also a field worthy of attention. At present, progress in this regard is still relatively limited.
Nanjing Innovation Investment Group
Combined with the industry you are engaged in, please introduce how large models are applied?
Chen Zhaohui
Manbang Group is China’s largest road freight matching platform for trucks and cargo, accumulating a large amount of data in transportation scenarios, covering five aspects: cargo, trucks, roads, people, and venues. Different from the direct dispatch mechanism of passenger transportation platforms such as Didi, the completion of freight orders involves a longer communication process. At present, the average unit price of road freight orders is about 2,000 yuan. During the transaction process, shippers and truck owners have more complex decision-making processes, so the frictional costs of transactions are correspondingly higher.
According to Manbang’s internal statistics, truck owners pay approximately 200 yuan in discovery costs for the information of each freight order, and shippers bear about 40 yuan in scheduling costs for each freight order—with considerable time expenditure included. Therefore, starting from improving communication efficiency and reducing industry transaction costs, Manbang is developing an intelligent driver assistant and an intelligent shipper assistant based on large language models. These assistants will help both parties with information matching and interaction in the pre-transaction stage to improve matching efficiency. Based on an estimate of 2 billion annual road passenger transport orders, this single measure is expected to reduce costs by 100 billion yuan for the industry.
Nanjing Innovation Investment Group
What prospects do you have for the future development trend of large models?
Chen Zhaohui
From the perspective of technical capabilities, the cycle of mutual promotion and development between large models and computing power has been initially verified. In the future, with the input of more high-quality, structured data, large models may usher in a leap in capabilities that exceeds human expectations.
In terms of the model layer, by the end of this year, the capabilities of several mature leading domestic general large model vendors will be able to reach the level of ChatGPT 3.5. It is expected that next year, there will be large models entering the stage of large-scale commercial application.
From the application perspective, at present, in this round of development of pre-trained large models, the products of domestic vendors at least have not yet reached their "iPhone moment" (a pivotal point for widespread adoption). Currently, in addition to reducing costs and improving efficiency and providing certain communication capabilities, the new demand space triggered by pre-trained large models is also highly anticipated.
In short, we should prepare mentally and in terms of capability management for the explosive growth of large model capabilities. Personally, I am very optimistic about the future capabilities of large models and the prospects of the industrial revolution they can trigger. At the same time, the ethics, jurisprudence, and safety standards of large models are also fields that deserve great attention.
Thank you for Dr. Chen Zhaohui’s sharing. We believe that feeding back to large models through user feedback and practical innovation will further consolidate the technology of basic large models and promote the scenario-based implementation of large models.
Source: Zhang Surong (Fourth Investment Department)
Reviewer: Xue Yao
Publisher: You Yi
