如何为chatGPT增加网络访问功能

问题背景

我们都知道chatGPT无法访问较新的数据，比如我希望询问硅谷银行的事情时，得到的回答是这样的:

当然如果使用newbing 可以达到获得我们想要的结果，那么除了newbing，我们能否自己为chatGPT增加网络访问功能呢？

解决方案

要解决这个问题，我们很容易想到需要搜索引擎这个工具，那么现在我们有2个工具1个是chatGPT，1个是搜索引擎, 接下来我们看什么样的流程可以完成我们的目标。

提供Context

如果我们可以在给chatGPT 提问时带上这些GPT所不知道的信息，chatGPT是可以给出不错的结果的。比如:

这些信息可以通过我们的工具搜索引擎获得，但这里的问题是如何得到搜索的关键字，因为我们并不知道GPT需要知道哪些信息。

让GPT自动生成要搜索的关键字

这里很好的1个思路是使用GPT产生要搜索关键字，这里应该是有借助GPT的推理能力，这也是最为关键的prompt

prompt:

'''Answer the following questions as best you can. 
You have access to the following tools:nnBing Search: 
A wrapper around Bing Search. Useful for when you need to answer questions 
about current events. Input should be a search query.nnUse the following 
format:nnQuestion: the input question you must answernThought: 
you should always think about what to donAction: the action to take, 
should be one of [Bing Search]nAction Input: the input to the actionn
Observation: the result of the actionn... (this Thought/Action/Action 
Input/Observation can repeat N times)nThought: 
I now know the final answernFinal Answer: 
the final answer to the original input questionnnBegin!nnQuestion: 
${问题}nThought:'''

当我们使用这个prompt时，GPT就会输出它需要知道的搜索关键字，也就是prompt中的 Action Input

然后我们将GPT产生的搜索关键字，使用搜索引擎API,查到最新的信息，最后将搜索引擎返回的最新结果和查询结果一同输入到GPT里面，得到最终结果。

这里注意，我们只是叙述1个最简模型，当我们询问的问题中有较多信息GPT不知道时，GPT可能会查询多次。

完整流程

画出我们的流程图:

实现

从0实现

按照这个思路我们使用cursor 写了1个nodejs express应用实现这个功能, 以下是服务端部分(此代码部分由cursor完成)。

const express = require('express');
const app = express();
const cors = require('cors');
const axios = require('axios');
app.use(cors());
app.use(express.json());

app.get('/test', (req, res) => {
 console.log('hello ai!');
  res.send('hello ai!');
});

app.post('/chat_ai_with_internal', async (req, res) => {
  //处理post请求
  const { query_message, pre_message } = req.body;
  if(!query_message || query_message === '') {
    console.log('error:', "queryMessage cannot be empty ");
    res.status(400).send({ error: "queryMessage cannot be empty" });
    return;
  }
  console.log('queryMessage:', query_message);
  let result = await getFinalAnswer(query_message, pre_message);
  result = await getAnswerFromGPT("Please translate the following statements into Chinese: " + result);
  console.log('result:', result);
  res.send({ role: "assistant", content: result});
});

async function getFinalAnswer(question, preMessage) {
  let answer = '';
  let searchResult = '';
  let prompt = '';
  while (!isFinalAnswer(answer)) {
    if(prompt === '') {
        prompt = questionToPrompt(question, searchResult, answer, preMessage);
    } else {
        prompt = handlePrompt(prompt, searchResult, answer);
    }
    answer = await getAnswerFromGPT(prompt);
    if (!isFinalAnswer(answer)) {
      const processedAnswer = await gptResToQuestion(answer);
      searchResult = await searchAPI(processedAnswer);
      console.log('searchResult:', searchResult);
    }
  }
  return answerHandler(answer);
}

function answerHandler(answer) {
  const regex = /Final Answer: (.*)/;
  const match = answer.match(regex);
  let result = '';
  if (match) {
    result = match[1];
  }
  return result;
}

async function gptResToQuestion(answer) {
  console.log('gptResToQuestion input:' +  answer);
  const regex = /Action Input: (.*)/;
  const match = answer.match(regex);
  let result = '';
  if (match) {
    result = match[1];
  }
  console.log('gptResToQuestion output:' + result);
  return result;
}

function handlePrompt(prompt, searchResult, lastAnswer) {
  if(searchResult && searchResult !== "" && lastAnswer && lastAnswer !== "") {
      prompt += lastAnswer + "nObservation: "  + searchResult + "nThought: ";
  }
  return prompt;
}

function questionToPrompt(question, searchResult, lastAnswer, preMessage) {
  if(preMessage && preMessage.length > 0) {
      question = "histroy message: " + JSON.stringify(preMessage) + "nnew question: " + question;
  }
  let prompt = "Answer the following questions as best you can. You have access to the following tools:nnBing Search: A wrapper around Bing Search. Useful for when you need to answer questions about current events. Input should be a search query.nnUse the following format:nnQuestion: the input question you must answernThought: you should always think about what to donAction: the action to take, should be one of [Bing Search]nAction Input: the input to the actionnObservation: the result of the actionn... (this Thought/Action/Action Input/Observation can repeat N times)nThought: I now know the final answernFinal Answer: the final answer to the original input questionnnBegin!nnQuestion: " + question+ "nThought: ";
  if(searchResult && searchResult !== "" && lastAnswer && lastAnswer !== "") {
      prompt += lastAnswer + "nObservation: "  + searchResult + "nThought: ";
  }
  return prompt;
}

function isFinalAnswer(answer) {
  return answer && answer.includes('Final Answer: ');
}

async function getAnswerFromGPT(prompt) {
  console.log("nnprompt is: " + prompt)
  const message = {role:"user", "content": prompt};
  const messageList = [message];
  console.log('messageList:', JSON.stringify(messageList));
  const OPEN_AI_KEY = "填入openaikey";
  const response = await axios.post('https://api.openai.com/v1/chat/completions', {
    messages: messageList,
    stop: ["nObservation:", "ntObservation:"],
    model: 'gpt-3.5-turbo',
  }, {
    headers: {
      'Content-Type': 'application/json',
      'Authorization': `Bearer ${OPEN_AI_KEY}`,
    },
  });
  console.log('getAnswerFromGPT result:', response.data.choices[0].message.content.trim());
  return response.data.choices[0].message.content.trim();
}

async function searchAPI(question) {
  if(!question || question === '') {
    throw new Error('Question cannot be empty');
  }
  const BING_SEARCH_API_KEY = "填入bing search key"
  const response = await axios.get('https://api.bing.microsoft.com/v7.0/search', {
    headers: {
      'Ocp-Apim-Subscription-Key': BING_SEARCH_API_KEY,
    },
    params: {
      q: question,
      count: 10,
      textDecorations: true,
      textFormat: "HTML",
    },
  });
  const snippets = response.data.webPages?.value.map(page => page.snippet);
  return snippets.join('');
}

app.listen(6666, () => {
  console.log('服务器已启动，本地地址：http://localhost:6666');
});

代码解释

cursor对以上代码自动生成的解释:

这段代码是一个基于express框架的node.js后端服务，主要提供了两个接口：/test和/chat_ai_with_internal。其中/test接口是一个get请求，返回”hello ai!”字符串；/chat_ai_with_internal接口是一个post请求，接收一个json格式的请求体，包含query_message和pre_message两个字段。其中query_message是一个字符串类型的参数，表示用户的问题；pre_message是一个数组类型的参数，表示之前的对话记录。接口的主要逻辑在getFinalAnswer函数中，该函数通过调用getAnswerFromGPT函数获取AI的回答，然后根据回答是否为最终答案来决定是否需要调用searchAPI函数进行搜索。如果回答不是最终答案，则需要将回答转化为问题，然后调用searchAPI函数进行搜索，获取搜索结果。最终答案的判断是通过判断回答中是否包含”Final Answer: “字符串来实现的。

效果展示

下面是效果，可以看到已经可以比较好的回答了

可以在命令行中看到GPT如何将自己不知道的信息转化未搜索关键字:

借助LangChain实现

我们是参考langchain来解析如何为chatGPT增加网络访问功能，其实我们如果直接使用langchain可以使用更加简洁的代码实现这个应用,以下是实现:

from flask import Flask,request
from flask_cors import CORS
import search
import json

app = Flask(__name__)
CORS(app)

@app.route('/test')
def index():
    return 'Hello, ai!'

@app.route('/chat_ai_with_internal', methods=["POST"])
def searchPOST():
    data = json.loads(request.data);
    queryMessage = data['query_message'];
    preMessage = data['pre_message'];
    if len(queryMessage):
        res = {'role': 'assistant', 'content': search.query(queryMessage, preMessage)};
        return json.dumps(res);
    return "search error"

if __name__ == '__main__':
    app.run(port=8115)

import os
os.environ["BING_SUBSCRIPTION_KEY"] = "填入 search key"
os.environ["BING_SEARCH_URL"] = "https://api.bing.microsoft.com/v7.0/search"
os.environ['OPENAI_API_KEY'] = "填入openai key"

from langchain.utilities import BingSearchAPIWrapper
search = BingSearchAPIWrapper(k=5)

from langchain.agents import load_tools
from langchain.agents import initialize_agent

from langchain.llms import OpenAIChat
llm = OpenAIChat()

tool_names = ["bing-search"]
tools = load_tools(tool_names)
agent = initialize_agent(tools, llm, agent="zero-shot-react-description", verbose=True)

def query(queryKey:str, preMessage: str) -> str:
    print("queryKey is %s", queryKey);
    print("preMessage is %s", preMessage)
    if len(preMessage):
        llm.prefix_messages = preMessage
    res = agent.run(queryKey);
    print(res);
    translationRes = llm.generate(["请将下面语句翻译为中文:" + res]).generations[0][0].text
    return  translationRes;

效果基本是一致的：

注：

1. 本文参考LangChain实现，prompt也是参考LangChain。
2. 本文大部分代码是借助cursor自动生成。

结尾

AI大模型时代已来，一定会对我们的生活带来极大的改变，无论是底层的模型层，还是如何使用这些底层模型的应用层都有很多事情值得我们研究，LangChain就为我们在使用大模型上提供了很好的启示。