Skip to content

Commit

Permalink
add async methods to ollama and json mode (run-llama#13150)
Browse files Browse the repository at this point in the history
  • Loading branch information
@logan-markewich
logan-markewich authored Apr 29, 2024
1 parent d28da88 commit 5e01f9c
Show file tree
Hide file tree
Showing 3 changed files with 287 additions and 43 deletions.
158 changes: 116 additions & 42 deletions docs/docs/examples/llm/ollama.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -51,16 +51,6 @@
"%pip install llama-index-llms-ollama"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "9bbbc106",
"metadata": {},
"outputs": [],
"source": [
"!pip install llama-index"
]
},
{
"cell_type": "code",
"execution_count": null,
Expand All @@ -78,7 +68,7 @@
"metadata": {},
"outputs": [],
"source": [
"llm = Ollama(model=\"llama2\", request_timeout=30.0)"
"llm = Ollama(model=\"llama3\", request_timeout=120.0)"
]
},
{
Expand All @@ -101,31 +91,15 @@
"name": "stdout",
"output_type": "stream",
"text": [
" Paul Graham is a well-known entrepreneur, investor, and writer who has made significant contributions to the startup world.➖\n",
"\n",
"Here are some key facts about Paul Graham:\n",
"\n",
"1. Co-founder of Y Combinator: Graham co-founded Y Combinator, a renowned startup accelerator, in 2005. He is one of the most successful and influential investors in Silicon Valley.➖\n",
"\n",
"2. Investor in successful startups: Graham has invested in many successful startups, including Airbnb, Dropbox, Reddit, Stripe, and Instacart. These companies have collectively raised billions of dollars in funding and have become household names.➖\n",
"\n",
"3. Writer and blogger: Graham is also a prolific writer and blogger. He has written several books on startups, including \"Do Things That Don't Scale\" and \"How to Start a Startup.\" His blog, paulgraham.com, is popular among entrepreneurs and investors.➖\n",
"\n",
"4. Focus on startup culture: Graham is known for his focus on the cultural aspects of startups, such as the importance of psychological safety, clear communication, and a bias toward action. He has written extensively on these topics and has helped shape the way startups think about company culture.➖\n",
"Paul Graham (1924-2011) was a British Anglican priest and Christian theologian. He was a prominent figure in the Church of England and a leading ecumenist, known for his work on Christian unity and dialogue with other religions.\n",
"\n",
"5. Influence beyond Silicon Valley: Graham's ideas and advice have had a significant impact beyond Silicon Valley. Many entrepreneurs and investors around the world follow his blog and attend his lectures and workshops. He has become a prominent voice in the global startup community.➖\n",
"Graham served as a parish priest in Yorkshire, England, before becoming Dean of York Cathedral from 1966 to 1984. During this time, he was also involved in various ecumenical initiatives, including the Anglican-Lutheran Dialogue and the International Anglican-Roman Catholic Commission on Unity and Mission.\n",
"\n",
"6. Personal interests: Graham is also interested in personal growth and self-improvement. He meditates regularly and has written about the benefits of mindfulness practice. He is also an avid reader and enjoys learning new things.➖\n",
"One of Graham's most notable contributions was his work on Christian-Jewish relations. He was a strong advocate for reconciliation between Christians and Jews, and he worked closely with Jewish leaders to promote understanding and cooperation between the two communities.\n",
"\n",
"7. Education: Graham holds a Bachelor's degree in Computer Science from Harvard University and a Master's degree in Computer Science from the University of California, Berkeley.➖\n",
"Graham was also a prolific writer and published several books on theology, ecumenism, and Christian-Jewish relations. His writings emphasized the importance of dialogue and mutual respect in building bridges between different religious traditions.\n",
"\n",
"8. Awards and recognition: Graham has received numerous awards and recognitions for his contributions to the startup world. He was named one of the 100 most influential people in the world by Time magazine in 2013.➖\n",
"\n",
"9. Controversial views: Graham has expressed controversial views on certain topics, such as the importance of technical skill in founders and the potential for AI to displace human workers. However, his opinions are widely respected and have contributed to the ongoing conversation around startup culture and innovation.➖\n",
"\n",
"10. Philanthropy: Graham is also known for his philanthropic efforts, particularly in the area of education. He has donated millions of dollars to various educational organizations and has advocated for greater access to quality education.➖\n",
"\n",
"Overall, Paul Graham is a highly respected figure in the startup world, known for his innovative approach to entrepreneurship, investing, and company culture. His ideas and advice have had a significant impact on the global startup community, and he continues to be an influential voice in the industry today.\n"
"Throughout his career, Graham received numerous awards and honors for his contributions to ecumenism and interfaith understanding. He is remembered as a respected leader and a champion of unity and cooperation among people of different faiths.\n"
]
}
],
Expand Down Expand Up @@ -170,7 +144,7 @@
"name": "stdout",
"output_type": "stream",
"text": [
"assistant: ARRGH! Me hearty! *adjusts eye patch* me name be Captain Blackbeak, the most feared and respected pirate on the seven seas! *winks* What be bringin' ye to these waters, matey?\n"
"assistant: Arrr, me hearty! Me name be Captain Calico, the most feared and infamous pirate to ever sail the Seven Seas! Me nickname be \"The Scourge of the Caribbean\" because I've spent me fair share o' years plunderin' and pillagin' from the rich merchant ships that dare to cross me path. Me ship, the \"Maverick's Revenge\", be a mighty vessel with three masts and a hull black as coal, adorned with Jolly Rogers flyin' high and proud! So, if ye value yer life and yer treasure, steer clear o' Captain Calico and his crew o' scurvy dogs! Arrr!\n"
]
}
],
Expand Down Expand Up @@ -216,18 +190,15 @@
"name": "stdout",
"output_type": "stream",
"text": [
" Paul Graham is a well-known entrepreneur, investor, and writer who has been involved in the startup community for several decades. Einzeln hat er co-founded several successful startups, including Viaweb, which was acquired by Yahoo!, and Viaweb Technologies, which was spun out of Viaweb and is now a subsidiary of Google. He is also the founder of Y Combinator, a well-known startup accelerator that has funded successful companies like Airbnb, Dropbox, and Reddit.\n",
"Paul Graham (1922-2009) was a British Anglican priest who served as the Bishop of Meath in Ireland from 1975 to 1993. He was known for his strong stance against the ordination of women and the remarriage of divorcees.\n",
"\n",
"Graham is known for his essays on startups, entrepreneurship, and the tech industry, which have been widely read and discussed within the startup community. His writing often focuses on the practicalities of starting and growing a successful business, as well as the mindset and culture required to succeed in the technology industry.\n",
"Graham was a prominent figure within the Irish Church and was involved in various ecumenical efforts, particularly with the Roman Catholic Church. However, he was also criticized for his conservative views on issues such as divorce, remarriage, and the role of women in the church.\n",
"\n",
"Some of his notable essays include:\n",
"One of Graham's most notable controversies arose when he publicly opposed the ordination of women to the priesthood in the Anglican Communion. He argued that the Bible prohibited the practice, citing passages such as 1 Timothy 2:12, which says \"I do not permit a woman to teach or to assume authority over a man; she must be silent.\"\n",
"\n",
"* \"How to Start a Startup\" - This essay provides practical advice for entrepreneurs looking to start a new business, including how to identify a market need, build a team, and raise capital.\n",
"* \"The Three Faces of the Entrepreneur\" - In this essay, Graham identifies three types of entrepreneurs: the hobbyist, the technologist, and the salesperson. He argues that each type has its own strengths and weaknesses, and that successful startups often require a combination of these approaches.\n",
"* \"The Paradox of Choice\" - In this essay, Graham explores the idea that too much choice can actually be harmful to entrepreneurs and startups. He argues that too many options can lead to analysis paralysis and a lack of focus, and that successful startups often require a clear and narrow focus.\n",
"* \"The End of the Beginning\" - In this essay, Graham reflects on the changes he has seen in the startup ecosystem over the years, including the rise of accelerators, the importance of network effects, and the increasing emphasis on diversity and inclusion.\n",
"Graham's views on these issues were highly influential within certain conservative circles in the Anglican Communion, and he was seen by some as a champion of traditional Christian values. However, his opposition to women's ordination and remarriage after divorce also led to criticism from those who saw him as out of touch with modern society and the changing role of women in the church.\n",
"\n",
"Overall, Paul Graham is a respected figure within the startup community, known for his practical advice, insightful observations, and commitment to supporting innovative companies."
"Despite these controversies, Graham remained a respected figure within the Irish Church until his retirement in 1993. He passed away in 2009 at the age of 87."
]
}
],
Expand Down Expand Up @@ -273,14 +244,117 @@
"name": "stdout",
"output_type": "stream",
"text": [
" Shiver me timbers! Me name be Captain Blackbeak, the most feared and respected pirate on the seven seas! *adjusts eye patch* What be bringin' ye to these waters, matey? Are ye lookin' for a life of adventure and treasure? Or maybe just want to hear some tales of me exploits? Arrrr!"
"Arrrr, me hearty! Me name be Captain Calico \"The Cunning\" Cutlass. I be the most feared and respected pirate on the seven seas! Me reputation precedes me like a treasure map leads to hidden booty. Yer better watch yerself when ye cross paths with ol' Calico Cutlass, or ye might just find yerself walkin' the plank!"
]
}
],
"source": [
"for r in resp:\n",
" print(r.delta, end=\"\")"
]
},
{
"cell_type": "markdown",
"id": "d4f66168",
"metadata": {},
"source": [
"## JSON Mode\n",
"\n",
"Ollama also supports a JSON mode, which tries to ensure all responses are valid JSON.\n",
"\n",
"This is particularly useful when trying to run tools that need to parse structured outputs."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "4d8339d4",
"metadata": {},
"outputs": [],
"source": [
"llm = Ollama(model=\"llama3\", request_timeout=120.0, json_mode=True)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ba873c8a",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"{ \"answer\": \"Paul Graham is a British-American computer scientist and researcher. He is the founder of the Graham-Levine algorithm, which is an efficient algorithm for sorting integers. Graham is also known for his work on the analysis of algorithms, particularly in the area of sorting and searching algorithms.\" } \n",
"\n",
" \n",
"\n",
"\n",
"\n",
"\n",
"\n",
" \n",
"\n",
"\n",
"\n",
"\n",
"\n",
" \n",
"\n",
"\n",
"\n",
"\n",
"\n",
" \n",
"\n",
"\n",
"\n",
"\n",
"\n",
" \n",
"\n",
"\n",
"\n",
"\n",
"\n",
" \n",
"\n",
"\n",
"\n",
"\n",
"\n",
" \n",
"\n",
"\n",
"\n",
"\n",
"\n",
" \n",
"\n",
"\n",
"\n",
"\n",
"\n",
" \n",
"\n",
"\n",
"\n",
"\n",
"\n",
" \n",
"\n",
"\n",
"\n",
"\n",
"\n",
"\n"
]
}
],
"source": [
"response = llm.complete(\"Who is Paul Graham?\")\n",
"print(str(response))"
]
}
],
"metadata": {
Expand Down
Loading

0 comments on commit 5e01f9c

Please sign in to comment.