# run this as %%bash cell or in terminal
conda create --name llm
conda activate llm
pip install langchain
pip install openai
pip install duckduckgo-search
# for notebook kernel (I always forget this)
conda install ipykernel
ipython kernel install --user --name=llmIntro to LangChain Output Parsers
If you have fiddled around with ChatGPT or LangChain yet, you would have realised how difficult it is to get the outcome we expect everytime. For whatever reason, we might expect a list, or a table and it would output a lot of text that we never asked for.
Let’s follow an example to see what is the issue we usually face with LLMs and how we can use Output Parsers to resolve it. But first, some setup!
Basic Setup
import os
# yes you need to get the openAI key
os.environ["OPENAI_API_KEY"] = ""
# need to make a free account at https://serper.dev/ for this key
os.environ["SERPER_API_KEY"] = ""Back to Our Example
Suppose you want to create a LLM chain to suggest a dinner restaurant along with details like it’s address, a short description and the most popular dish. Let’s see what happens when we try to do that.
Ideally I’d like to have langchain search it online and then hook up with the right chain, but let’s try to keep things simple for now.
from langchain.prompts import ChatPromptTemplate, HumanMessagePromptTemplate
from langchain.chat_models import ChatOpenAI
from langchain.chains import LLMChain
template_string = """You are a travel agent who helps guests with popular
food places for their favourite cuisines. You are helping a guest who
wants to eat {cuisine} food in {city}. Recommend a restaurant for them.
Output the name of the restuarant, address, a single short sentence
description, and the most popular dish. Return the output in a JSON format."""
prompt = ChatPromptTemplate(
messages=[
HumanMessagePromptTemplate.from_template(template_string)
],
input_variables=["cuisine", "city"],
)
food_llm = ChatOpenAI()
food_chain = LLMChain(prompt=prompt, llm=food_llm)
response = food_chain.run(cuisine="Chinese", city="New York")
print(response){
"restaurant_name": "Joe's Shanghai",
"address": "9 Pell St, New York, NY 10013",
"description": "Joe's Shanghai is a bustling restaurant known for its authentic Chinese cuisine and lively atmosphere.",
"popular_dish": "Soup Dumplings"
}Now this response seems like a very nice JSON, but it is actually a string with formatting like below
response'{\n "restaurant_name": "Joe\'s Shanghai",\n "address": "9 Pell St, New York, NY 10013",\n "description": "Joe\'s Shanghai is a bustling restaurant known for its authentic Chinese cuisine and lively atmosphere.",\n "popular_dish": "Soup Dumplings"\n}'which makes it a nightmare to parse. And God know what different formats LLM would output, each requiring it’s own parser. LOL no way.
Hello Structured Output Parser
We will cover one of the simpler output parsers called Structure Output Parser.It is used to make sure that the output of the model is a data structure with text fields. Let’s see it in action!
We need the below ingredients - a ResponseSchema for each of the text fields we want in our output - put all response schemas together into an array - create a StructureOutputParser object with the above array - this will create format instructions to guide the LLM towards the expected output
from langchain.output_parsers import StructuredOutputParser, ResponseSchema
from langchain.prompts import (PromptTemplate,
ChatPromptTemplate,
HumanMessagePromptTemplate)
from langchain.llms import OpenAI
from langchain.chat_models import ChatOpenAI
# create a response schema
response_schemas = [
ResponseSchema(name="restaurant_name",
description="Name of the restaurant suggested to the user"),
ResponseSchema(name="address",
description="address of the suggested restaurant"),
ResponseSchema(name="description",
description="a short description of the restaurant"),
ResponseSchema(name="popular_dish",
description="a popular dish at the restaurant"),
]
output_parser = StructuredOutputParser.from_response_schemas(response_schemas)
format_instructions = output_parser.get_format_instructions()
print(format_instructions)The output should be a markdown code snippet formatted in the following schema, including the leading and trailing "```json" and "```":
```json
{
"restaurant_name": string // Name of the restaurant suggested to the user
"address": string // address of the suggested restaurant
"description": string // a short description of the restaurant
"popular_dish": string // a popular dish at the restaurant
}
```now let’s plug this into our chain above!
template_string = """You are a travel agent who helps guests with popular
food places for their favourite cuisines. You are helping a guest who
wants to eat {cuisine} food in {city}. Recommend a restaurant for them.
Output the name of the restuarant, address, a single short sentence
description, and the most popular dish.
{format_instructions}"""
prompt = ChatPromptTemplate(
messages=[
HumanMessagePromptTemplate.from_template(template_string)
],
input_variables=["cuisine", "city"],
partial_variables={"format_instructions": format_instructions}
)
food_llm = ChatOpenAI()
food_chain = LLMChain(prompt=prompt, llm=food_llm)
response = food_chain.run(cuisine="Chinese", city="New York")
print(response)```json
{
"restaurant_name": "Nom Wah Tea Parlor",
"address": "13 Doyers St, New York, NY 10013",
"description": "A historic dim sum parlor serving traditional Chinese dishes in Chinatown.",
"popular_dish": "Shrimp and snow pea leaf dumplings"
}
```# and now we parse the output
output_parser.parse(response){'restaurant_name': 'Nom Wah Tea Parlor',
'address': '13 Doyers St, New York, NY 10013',
'description': 'A historic dim sum parlor serving traditional Chinese dishes in Chinatown.',
'popular_dish': 'Shrimp and snow pea leaf dumplings'}Yay! We have successfully parsed the output of the LLM model and avoided writings them ourselves. So much for being lazy! (but this helps with a lot of handcrafting and rule writing, phew)
CHAIN IT UP!
I mean, what’s the point of writing chains if I can’t put the output parser in chain. Let’s go!
To achieve this, we simply add output_parser=output_parser to our chain and voila
prompt = ChatPromptTemplate(
messages=[
HumanMessagePromptTemplate.from_template(template_string)
],
input_variables=["cuisine", "city"],
partial_variables={"format_instructions": format_instructions},
)
food_llm = ChatOpenAI()
food_chain = LLMChain(prompt=prompt, llm=food_llm, output_parser=output_parser)
response = food_chain.run(cuisine="Chinese", city="New York")
response{'restaurant_name': "Joe's Shanghai",
'address': '9 Pell St, New York, NY 10013',
'description': 'A popular Chinese restaurant known for its soup dumplings.',
'popular_dish': 'Soup Dumplings'}Conclusion
We covered the simplest OutputParser in this blog and show how to use it for nuanced control over output. It helps keep templates clean, and avoid writing unnecessary parsers. I expect these to still fail sometimes (still LLMs after all) but they are much more robust than the previous approach.
What Next?
But LangChain is SO MUCH more powerful! The way more cooler parsers is the Pydantic parser. It is a bit more involved, but it is worth it. We will cover it in the next blog. Till then, keep chaining! ⛓️ Cheers! 🫖
References
Bonus
