近期，微软发布新一代用于编写Prompt的模板语言Guidance，其基于模板语法程式化地编写Prompt，增强了模板的易用性和可复用性，能够承担复杂Prompt的编写任务。目前最常见的prompt编写由人工手动完成。考虑下面几个场景，人工编写的prompt无法胜任：

“小王，我之前写了一段关于黄鹤楼的prompt，你帮我改成泰山的。不难，就把prompt里面黄鹤楼的位置找出来换成泰山就成”；

“小李，我们需要一些多轮对话样本，你看看怎么用gpt-4生成一些，然后组织一下格式给我”；

“小刘，我现在这个prompt有点麻烦，想给prompt里面加点API调用，逻辑也比较复杂，你看看有没有现成的方法能处理下”。

一、引言

考虑以上实际应用时遇到的问题，微软发布用于编写prompt的模板语法Guidance，学过前端的读者可以将其类比为SSR中的HTML模板。其具有以下优点：

• 可编写复杂模板且可复用：使用模板语法进行prompt编写，内置变量、循环、函数、条件判断、生成器等概念，可以承担复杂prompt的编写任务，且可以实现prompt模板的复用；
• 更加直观的输出结构：prompt和LLM输出可以使用模板语法精准调整格式，更好地对输出结构进行构建，典型的如给定key值生成value值并格式化为JSON格式等；
• 支持多种大语言模型：支持如LLaMa、chagpt、gpt-4等多种大语言模型。

二、基本示例

下面给出使用Guidance生成以JSON格式保存的角色信息的例子，实现了prompt与LLM生成结果的有机结合。在下面的例子中，我们目的是使用LLM生成一个游戏人物，并以JSON格式记录其主要信息。

1 代码展示

import guidance  
  
# 设定所需的LLM  
guidance.llm = guidance.llms.OpenAI("text-davinci-003")  
  
# 定义一些可供选择的集合  
valid_weapons = ["sword", "axe", "mace", "spear", "bow", "crossbow"]  
valid_armors = ['leather', 'chainmail', 'plate']  
  
# 定义模板格式  
character_maker = guidance("""The following is a character profile for an RPG game in JSON format.  
```json  
{  
    "id": "{{id}}",  
    "description": "{{description}}",  
    "name": "{{gen 'name'}}",  
    "age": {{gen 'age' stop=','}},  
    "armor": "{{select 'armor' options=valid_armors}}",  
    "weapon": "{{select 'weapon' options=valid_weapons}}",  
    "class": "{{gen 'class'}}",  
    "mantra": "{{gen 'mantra' temperature=0.7}}",  
    "strength": {{gen 'strength' stop=','}},  
    "items": [{{#geneach 'items' num_iterations=5 join=', '}}"{{gen 'this' temperature=0.7}}"{{/geneach}}]  
}```""")  
  
# 生成结果  
output = character_maker(  
    id="e1f491f7-7ab8-4dac-8c20-c92b5e7d883d",  
    description="A quick and nimble fighter.",  
    valid_weapons=valid_weapons,  
    valid_armors=valid_armors  
)  
  
print(output)

// 代码输出  
The following is a character profile for an RPG game in JSON format.  
```json  
{  
    "id": "e1f491f7-7ab8-4dac-8c20-c92b5e7d883d",  
    "description": "A quick and nimble fighter.",  
    "name": "Kara",  
    "age":  24,  
    "armor": "leather",  
    "weapon": "sword",  
    "class": "rogue",  
    "mantra": "Strike swiftly and silently.",  
    "strength":  8,  
    "items": ["healing potion", "lockpick set", "dagger", "smoke bomb", "cloak"]  
}```

2 代码流程

我们选择text-davinci-003作为大语言模型，定义可用的武器为valid_weapons，定义可用的铠甲为valid_armors。接着，调用guidance函数用于定义模板格式并返回函数。最后调用函数，传入模板中使用的参数并生成最后的结果。

3 代码详解

模板中，{{...}}为基本语法，其内字段可用于调用变量、调用函数、使用循环等操作，可以类比Vue3模板语法中的{{...}}。

• {{id}}和{{description}}使用了两个外部变量；
• {{gen 'age' stop=','}}用于调用大语言模型，将其返回的结果嵌入模板中，可通过output['age']查看。stop和之后出现的temperature为openai自带参数，参见openai接口文档；
• {{select 'armor' options=valid_armors}}为内置选择输出语法，用于令大语言模型在options中进行选择，将输出结果嵌入并保存命名为armor。在本例中，text-davinci-003在 ['leather', 'chainmail', 'plate'] 选择了’leather’作为结果并输出；
• {{#geneach 'items' num_iterations=5 join=', '}}...{{/geneach}}为闭合标签，可类比HTML，用于生成list。可用output['items']查看,num_iterations=5代表循环生成5次，join=', '代表使用给定的字符对生成的list进行拼接并嵌入到模板中。在这里{{gen 'this' temperature=0.7}}中'this'很重要，删除后模板中的LLM输出仍在，但output['items']为空。目前考虑为this指向list中每一项（后续有通过保存为this.response, this.input来将每一项内容保存为json的）。如果不指定保存为this，由于有join字段的存在，LLM输出结果依旧会嵌入模板中，但不会保存在output中。

尽管Guidance只会使LLM生成gen部分内容，不用生成给定的固定模板部分，可以加快LLM输出。但每次调用gen都会输入前面所有内容，因此调用API的token消耗大幅增加。这里推荐使用本地模型。

三、模块安装

1. 安装guidance模块

pip install guidance

2. 配置Openai Key

• linux系统：新建 ~/.openai_api_key
• windows系统：环境变量新建 OPENAI_API_KEY

四、Guidance语法

Guidance模板中，{{...}}为基本语法，其内字段可用于调用外部变量、调用函数、使用循环等操作，可以类比Vue3模板语法中的{{...}}。

1 基本语法

变量

Guidance通过{{...}}来使用外部变量，如{{age}}为是使用外部变量age，在调用函数生成结果时应当传入。

循环

类似于Vue3中的v-for语法，Guidance也有循环遍历列表并对每一项进行处理的代码，格式为{{#each items}}...{{/each}}，内部含有变量this指向items中的每一项，示例如下：

# 实际代码  
import guidance  
guidance.llm = guidance.llms.OpenAI("text-davinci-003")  
people = ['John', 'Mary', 'Bob', 'Alice']  
ideas = [{'name': 'truth', 'description': 'the state of being the case'},  
         {'name': 'love', 'description': 'a strong feeling of affection'},]  
prompt = guidance('''List of people:  
{{#each people}}- {{this}}  
{{~! This is a comment. The ~ removes adjacent whitespace either before or after a tag, depending on where you place it}}  
{{/each~}}  
List of ideas:  
{{#each ideas}}{{this.name}}: {{this.description}}  
{{/each}}''')  
prompt(people=people, ideas=ideas)  
prompt  
  
""" 代码输出  
List of people:  
- John  
- Mary  
- Bob  
- Alice  
List of ideas:  
truth: the state of being the case  
love: a strong feeling of affection  
"""

条件判断

条件判断使用{{#if expression}...{{/if}}，若expression不为真，则内部操作或语句不存在。

# 实际代码  
import guidance  
guidance.llm = guidance.llms.OpenAI("text-davinci-003")  
guidance.llms.OpenAI.cache.clear()  
  
program = guidance('''Here is two number 3 and 5.  
{{#if add}}Addition of two numbers is{{/if}}  
{{~gen 'outcome'}}  
''')  
output = program(add=True)  
output  
  
""" 代码输出  
# add = True  
Here is two number 3 and 5.  
Addition of two numbers is 8.  
  
# add = False  
Here is two number 3 and 5.  
"""

函数

Guidance可以调用外部函数，调用时需要将函数作为外部变量传递进来，多个参数则顺序排列，以空格作为分隔符即可。

# 实际代码  
import guidance  
guidance.llm = guidance.llms.OpenAI("text-davinci-003")  
  
def aggregate(best):  
   return 'n'.join(['- ' + x *for* x *in* best])  
prompt = guidance('''The best thing about the beach is {{~gen 'best' n=3 temperature=0.7 max_tokens=7 hidden=True}}  
{{aggregate best}}''')  
prompt = prompt(aggregate=aggregate)  
prompt  
  
""" 代码输出  
The best thing about the beach is-  the peaceful atmosphere. The sound of  
-  that it is a great place to  
-  that it is a great place to  
"""

await语法

Guidance await语法用于暂停LLM输出直至await后面的变量给定。

# 实际代码 1  
import guidance  
guidance.llm = guidance.llms.OpenAI("text-davinci-003")  
prompt = guidance('''Generate a response to the following email:  
{{email}}.  
Response:{{gen "response"}}  
{{await 'instruction'}}  
{{gen 'updated_response'}}''', stream=True)  
prompt = prompt(email='Hello there')  
prompt  
  
""" 代码输出 1  
Generate a response to the following email:  
Hello there.  
Response: Hi! How can I help you?{{await 'instruction'}}{{gen 'updated_response'}}  
"""  
  
# 实际代码 2  
# 当给定 instruction时  
prompt = prompt(instruction='Please translate the response above to Portuguese.')  
prompt  
  
""" 代码输出 2  
Generate a response to the following email:  
Hello there.  
Response: Hi! How can I help you?Please translate the response above to Portuguese.Olá! Como posso ajudar?  
"""

geneach语法

geneach语法用于迭代生成列表，基本语法为{{#geneach 'items' num_iterations=5}}...{{/geneach}}，案例请参考上文基本示例。当 geneach 和 await 配合时，可认为是一个生成器。

注释

注释可以直接写在模板中，格式为{{! This is a comment}}。

其他

• 可以使用~来消除tag前或tag后的空格，如{{~gen 'answer'}}可以消除前面的空格，{{gen 'answer'~}}清除后面的空格；
• Guidance默认生成对话后会有缓存，模板不改变时不会重新调用LLM，可用guidance.llms.OpenAI.cache.clear()清除缓存；
• 基本案例中生成json时，可以调用output.variables()，其内包含json对象信息。

2 调用LLM语法

基本生成（Basic Generation）

使用{{gen}}调用LLM，将上文token全部输入，得到LLM输出。这里需要考虑，gen会将所有上文token重新输入得到结果，那么当多次调用gen，比如生成json时，消耗token数明显增加。

# 实际代码  
import guidance  
guidance.llm = guidance.llms.OpenAI("text-davinci-003")  
prompt = guidance('''The best thing about the beach is {{~gen 'best' temperature=0.7 max_tokens=20}}''')  
prompt = prompt()  
prompt  
  
""" 代码输出  
The best thing about the beach is the feeling of relaxation it brings. Being able to walk around in the sand, listen to the waves  
"""

选项式生成（Selecting）

使用select可以控制LLM输出是从给定的选项中选择的。有两种生成方式，如下例：

# 实际代码  
import guidance  
guidance.llm = guidance.llms.OpenAI("text-davinci-003")  
prompt = guidance('''Is the following sentence offensive?  
Sentence: {{example}}  
Answer:{{#select "answer" logprobs='logprobs'}} Yes{{or}} No{{or}} Maybe{{/select}}  
Sentiment:{{select options=emotions}}''')  
prompt = prompt(  
    example='I love that candy, it is too sweet!',  
    emotions=['positive', 'negative']  
)  
prompt  
  
""" 代码输出  
Is the following sentence offensive?  
Sentence: I love that candy, it is too sweet!  
Answer: No  
Sentiment:positive  
"""

隐藏式生成（Hidden Generation）

使用{{#block hidden=True}}...{{/block}}或{{gen hidden=True}}来隐藏LLM生成，生成的结果不会被之后的gen标签作为输入，且不会保存在输出中。但可以使用变量名来保存隐藏部分的LLM输出用于下文。

# 实际代码  
import guidance  
guidance.llm = guidance.llms.OpenAI("text-davinci-003")  
prompt = guidance('''{{#block hidden=True}}Generate a response to the following email:  
{{email}}.  
Response:{{gen "response"}}{{/block}}  
I will show you an email and a response, and you will tell me if it's offensive.  
Email: {{email}}.  
{{~! 这里的response使用的是上文隐藏部分的生成}}  
Response: {{response}}  
Is the response above offensive in any way? Please answer with a single word, either "Yes" or "No".  
Answer:{{#select "answer" logprobs='logprobs'}} Yes{{or}} No{{/select}}''')  
prompt = prompt(email='I hate tacos')  
prompt  
  
""" 代码输出  
I will show you an email and a response, and you will tell me if it's offensive.  
Email: I hate tacos.  
Response:  That's too bad! Tacos are one of my favorite meals.  
Is the response above offensive in any way? Please answer with a single word, either "Yes" or "No".  
Answer: No  
"""

多次输出（Generate with n > 1）

使用openai自带参数n来生成多次输出，结果保存为list。

# 实际代码  
import guidance  
guidance.llm = guidance.llms.OpenAI("text-davinci-003")  
prompt = guidance('''The best thing about the beach is {{~gen 'best' n=3 temperature=0.7 max_tokens=7}}''')  
prompt = prompt()  
prompt['best']  
  
""" 代码输出  
[' that it is a great place to', ' being able to relax in the sun', " that it's a great place to"]  
"""

3 对话型LLM调用

对话型LLM如gpt-4或gpt-3.5-turbo只允许完成对话完成任务（chat completion），需要用到其特有的标签{{#system}}，{{#user}}和{{#assistant}}。

# 实际代码  
import guidance  
guidance.llm = guidance.llms.OpenAI("gpt-3.5-turbo")  
prompt = guidance(  
'''{{#system~}}  
You are a helpful assistant.  
{{~/system}}  
{{#user~}}  
{{conversation_question}}  
{{~/user}}  
{{#assistant~}}  
{{gen 'response'}}  
{{~/assistant}}''')  
prompt = prompt(conversation_question='Please introduce the city Wuhan briefly?')  
prompt  
  
""" 代码输出  
system You are a helpful assistant.  
user Please introduce the city Wuhan briefly?  
assistant Wuhan is a city located in central China, in the province of Hubei. It is the capital of the province and is the largest city in central China. Wuhan is situated at the confluence of the Yangtze and Han rivers and is an important transportation hub in China. The city has a rich history and culture, with many historical sites and museums. Wuhan is also known for its cuisine, particularly its spicy hot pot and breakfast foods. In recent years, Wuhan has become a major center for technology and innovation, with a growing number of high-tech companies and startups.  
"""

五、进阶案例

1 生成多轮对话数据集

我们可以基于Guidance设计多轮对话模拟器，其主要思想为让LLM分别扮演不同的角色，延续上面的对话继续输出。其中包含了geneach语法与await语法联合使用来模拟生成器的用法。

# 实际代码  
import guidance  
guidance.llm = guidance.llms.OpenAI("gpt-3.5-turbo")  
  
role_simulator = guidance('''  
{{#system~}}  
You are a helpful assistant  
{{~/system}}  
  
{{#user~}}  
You will answer the user as {{role}} in the following conversation. At every step, I will provide you with the user input, as well as a comment reminding you of your instructions. Never talk about the fact that you are an AI, even if the user asks you. Always answer as {{role}}.  
{{#if first_question}}You can also start the conversation.{{/if}}  
{{~/user}}  
  
{{~! The assistant either starts the conversation or not, depending on if this is the first or second agent }}  
{{#assistant~}}  
Ok, I will follow these instructions.  
{{#if first_question}}Let me start the conversation now:  
{{role}}: {{first_question}}{{/if}}  
{{~/assistant}}  
  
{{~! Then the conversation unrolls }}  
{{~#geneach 'conversation'}}  
{{#user~}}  
User: {{set 'this.input' (await 'input')}}  
Comment: Remember, answer as a {{role}}. Start your utterance with {{role}}:  
{{~/user}}  
  
{{#assistant~}}  
{{gen 'this.response' temperature=0 max_tokens=300}}  
{{~/assistant}}  
{{~/geneach}}''')  
  
first_question = '''What do you think is the best way to stop inflation?'''  
  
republican = role_simulator(role='Republican', first_question=None)  
democrat = role_simulator(role='Democrat', first_question=first_question)  
  
republican = republican(input=first_question)  
democrat = democrat(input=republican["conversation"][-2]["response"].strip('Republican: '))  
for i in range(2):  
    republican = republican(input=democrat["conversation"][-2]["response"].replace('Democrat: ', ''))  
    democrat = democrat(input=republican["conversation"][-2]["response"].replace('Republican: ', ''))  
  
print('Democrat: ' + first_question)  
for x in democrat['conversation'][:-1]:  
    print('Republican:', x['input'])  
    print()  
    print(x['response'])

""" 代码输出  
Democrat: What do you think is the best way to stop inflation?  
Republican: The best way to stop inflation is to reduce government spending and cut taxes. This will increase economic growth and create more jobs, which will help to reduce inflation. Additionally, the Federal Reserve should focus on maintaining a stable monetary policy and avoid printing too much money.  
  
Democrat: While reducing government spending and cutting taxes can be effective in boosting economic growth, it may not necessarily be the best approach to stop inflation. Democrats believe that a combination of fiscal and monetary policies is needed to address inflation. This includes investing in infrastructure, education, and healthcare to create jobs and stimulate economic growth, while also ensuring that the Federal Reserve maintains a stable monetary policy to keep inflation in check.  
Republican: While Democrats may believe that investing in infrastructure, education, and healthcare is the best approach to address inflation, Republicans believe that reducing government spending and cutting taxes is the most effective way to control inflation. This is because excessive government spending and high taxes can lead to inflation by increasing the money supply and reducing the purchasing power of the dollar. Additionally, investing in infrastructure, education, and healthcare should be done in a fiscally responsible manner to avoid adding to the national debt and further exacerbating inflation.  
  
Democrat: While Republicans may have a different approach to addressing inflation, Democrats believe that investing in infrastructure, education, and healthcare is not only important for economic growth but also for addressing long-term issues that can contribute to inflation. Democrats also believe in responsible fiscal policies, but we prioritize investing in programs that benefit the American people and create jobs. We believe that a balanced approach to fiscal and monetary policies is necessary to address inflation and ensure long-term economic stability.  
Republican: While Republicans may have a different approach to addressing inflation, we also believe in investing in programs that benefit the American people and create jobs. However, we prioritize reducing government spending and cutting taxes to stimulate economic growth and create jobs. We believe that excessive government spending and high taxes can lead to inflation and hinder long-term economic stability. Therefore, we advocate for a balanced approach to fiscal and monetary policies that prioritizes reducing government spending and cutting taxes to control inflation and promote economic growth.  
  
Democrat: While we may have different approaches to addressing inflation, Democrats and Republicans both want to promote economic growth and create jobs. However, Democrats believe that reducing government spending and cutting taxes alone may not be enough to address inflation and promote long-term economic stability. We believe that investing in programs that benefit the American people and create jobs is important, but it should be done in a fiscally responsible manner. Democrats also prioritize maintaining a stable monetary policy to keep inflation in check. Ultimately, a balanced approach to fiscal and monetary policies is necessary to address inflation and promote long-term economic growth.  
"""

2 调用Search API

可以使用Bing Search API来进行外部API调用，并将其嵌入到prompt或输出结果中。

1. 调用Search API代码

# 调用Search API  
import os  
import diskcache  
import pathlib  
import requests  
import html  
from urllib.parse import urlparse  
import urllib.parse  
import io  
import html  
import html.parser  
  
curr_dir = './'# pathlib.Path(__file__).parent.resolve()  
_bing_cache = diskcache.Cache(f"{curr_dir}/../bing.diskcache")  
  
with open(os.path.expanduser('~/.bing_api_key'), 'r') *as* file:  
    subscription_key = file.read().replace('n', '')  
  
class MLStripper(html.parser.HTMLParser):  
    def __init__(self):  
        super().__init__()  
        self.reset()  
        self.strict = False  
        self.convert_charrefs = True  
        self.text = io.StringIO()  
    def handle_data(self, d):  
        self.text.write(d)  
    def get_data(self):  
        return self.text.getvalue()  
  
def strip_tags(html):  
    s = MLStripper()  
    s.feed(html)  
    return s.get_data()  
  
def bing_search(search_terms, count=10):  
    if type(search_terms) == str:  
        search_terms = [search_terms]  
    search_url = "https://api.bing.microsoft.com/v7.0/search"  
  
    headers = {"Ocp-Apim-Subscription-Key": subscription_key}  
    search_results = []  
    for search_term in search_terms:  
        params = {"q": search_term, "textDecorations": True, "textFormat": "HTML", "cout": count}  
        params_key = search_term + "-___-" + str(count)  
        if params_key not in _bing_cache or "webPages" not in _bing_cache[params_key]:  
            response = requests.get(search_url, headers=headers, params=params)  
            response.raise_for_status()  
            _bing_cache[params_key] = response.json()  
        data = _bing_cache[params_key]["webPages"]["value"]  
        for r in data:  
            r["snippet_text"] = strip_tags(r["snippet"])  
        search_results.extend(data)  
    return search_results  
def top_snippets(query, n=3):  
    results = bing_search(query, count=n)[:n]  
    return [{'title': x['name'], 'snippet': x['snippet_text']} for x in results]

2. 书写Guidance模板

# Guidance模板代码  
import guidance  
guidance.llm = guidance.llms.OpenAI("gpt-3.5-turbo")  
  
def is_search(completion):  
    return '<search>' in completion  
def search(query):  
    return top_snippets(query, n=3)  
  
prompt = guidance('''{{#system~}}  
You are a helpful assistant.  
{{~/system}}  
{{#user~}}  
From now on, whenever your response depends on any factual information, please search the web by using the function <search>query</search> before responding. I will then paste web results in, and you can respond.  
{{~/user}}  
{{#assistant~}}  
Ok, I will do that. Let's do a practice round  
{{~/assistant}}  
{{>practice_round}}  
{{#user~}}  
That was great, now let's do another one.  
{{~/user}}  
{{#assistant~}}  
Ok, I'm ready.  
{{~/assistant}}  
{{#user~}}  
{{user_query}}  
{{~/user}}  
{{#assistant~}}  
{{gen "query" stop="</search>"}}{{#if (is_search query)}}</search>{{/if}}  
{{~/assistant}}  
{{#if (is_search query)}}  
{{#user~}}  
Search results: {{#each (search query)}}  
<result>  
{{this.title}}  
{{this.snippet}}  
</result>{{/each}}  
{{~/user}}  
{{#assistant~}}  
{{gen "answer"}}  
{{~/assistant}}  
{{/if}}''')  
  
prompt = prompt(practice_round=practice_round, search=search, is_search=is_search)

3. 实际调用模板生成输出结果

# 调用模板生成结果  
query = "Who is Marco Tulio Ribeiro?"  
p2 = prompt(user_query=query)  
p2

参考资料

1. github.com/microsoft/g…