Gorilla Files

😍 v0.3 release 🚀

Highlights

⚡️ Released GoEx: A runtime that presents abstractions for safe execution of LLM generated code, APIs, actions, etc

🏆 Updates to Berkeley Function Calling Leaderboard (aka Berkeley Tool Calling Leaderboard) : Newer models including GPT-4o, gemini-flash and 1.5-pro, Hermes-2-Pro, etc. All measured along P95 and P99 latency, and costs besides accuracy.

What's Changed

• Fix Typos in Evaluation Script and System Prompt. Identify Errors in a Dataset by @zuxin666 in https://github.com/ShishirPatil/gorilla/pull/335
• BFCL April 8th Release by @HuanzhiMao in https://github.com/ShishirPatil/gorilla/pull/330
• Initial goex commit by @ShishirPatil in https://github.com/ShishirPatil/gorilla/pull/336
• BFCL April 9th Release (Dataset Bug Fix) by @HuanzhiMao in https://github.com/ShishirPatil/gorilla/pull/338
• BFCL April 10th Release (API Sanity Check) by @HuanzhiMao in https://github.com/ShishirPatil/gorilla/pull/339
• Add Support for NousResearch/Hermes-2-Pro-Mistral-7B Function Calling by @Fanjia-Yan in https://github.com/ShishirPatil/gorilla/pull/327
• Update raft.py with default p to match paper by @ShishirPatil in https://github.com/ShishirPatil/gorilla/pull/353
• GoEx Import Issues by @royh02 in https://github.com/ShishirPatil/gorilla/pull/354
• BFCL April 11th Patch. Add Latency Statistics. by @HuanzhiMao in https://github.com/ShishirPatil/gorilla/pull/347
• GoEx Gitignore User Credentials by @royh02 in https://github.com/ShishirPatil/gorilla/pull/344
• Fix Circular Import Issue for BFCL evluation pipeline by @HuanzhiMao in https://github.com/ShishirPatil/gorilla/pull/356
• Added Docker to README by @Noppapon in https://github.com/ShishirPatil/gorilla/pull/355
• [Bug fix] Add Hermes-2-Pro-Mistral-7B model to UNDERSCORE_TO_DOT to parse API properly by @JasonZhu1313 in https://github.com/ShishirPatil/gorilla/pull/364
• Update requirements.txt by @viniciuslazzari in https://github.com/ShishirPatil/gorilla/pull/343
• Fix script argument by @ricklamers in https://github.com/ShishirPatil/gorilla/pull/367
• BFCL April 16th Release by @HuanzhiMao in https://github.com/ShishirPatil/gorilla/pull/366
• Log error messages from API validation by @eitanturok in https://github.com/ShishirPatil/gorilla/pull/369
• Update .gitignore by @eitanturok in https://github.com/ShishirPatil/gorilla/pull/370
• BFCL April 18th Release (Pipeline only) by @HuanzhiMao in https://github.com/ShishirPatil/gorilla/pull/375
• Add missing argument to OSSHandler's _format_prompt function by @eitanturok in https://github.com/ShishirPatil/gorilla/pull/373
• Add FC + Prompt for Cohere command-r-plus by @harry-cohere in https://github.com/ShishirPatil/gorilla/pull/350
• BFCL April 19th Release (Dataset & Pipeline) by @HuanzhiMao in https://github.com/ShishirPatil/gorilla/pull/377
• Azure OpenAI support in raft.py by @cedricvidal in https://github.com/ShishirPatil/gorilla/pull/381
• BFCL April 25th Release (New Models) by @HuanzhiMao in https://github.com/ShishirPatil/gorilla/pull/386
• Colored logging configuration + displaying progress in logs by @cedricvidal in https://github.com/ShishirPatil/gorilla/pull/384
• BFCL April 27th Release (Bug Fix in Cost/Latency Calculation) by @HuanzhiMao in https://github.com/ShishirPatil/gorilla/pull/390
• BFCL April 28th Release (New Model: snowflake/arctic) by @Fanjia-Yan in https://github.com/ShishirPatil/gorilla/pull/397
• RAFT Recovery Mode for interruptions by @kaiwen129 in https://github.com/ShishirPatil/gorilla/pull/410
• Small corrections to possible_answers for simple test category by @aastroza in https://github.com/ShishirPatil/gorilla/pull/405
• BFCL May 6th Release (Dataset Bug Fix) by @HuanzhiMao in https://github.com/ShishirPatil/gorilla/pull/412
• RAFT DevContainer for GitHub Codespaces by @cedricvidal in https://github.com/ShishirPatil/gorilla/pull/379
• RAFT Add support for configuring separate completion and embedding endpoints + pytest by @cedricvidal in https://github.com/ShishirPatil/gorilla/pull/396
• RAFT Fix arbitrary code execution vulnerability in checkpoint feature by @cedricvidal in https://github.com/ShishirPatil/gorilla/pull/415
• handle parallel function calls from gemini by @vandyxiaowei in https://github.com/ShishirPatil/gorilla/pull/406
• RAFT Support for chat and completion model formats by @cedricvidal in https://github.com/ShishirPatil/gorilla/pull/417
• [RAFT] Edit encode prompt to include <ANSWER>: tag in label by @kaiwen129 in https://github.com/ShishirPatil/gorilla/pull/422
• [BFCL] Patch Gemini Handler by @HuanzhiMao in https://github.com/ShishirPatil/gorilla/pull/421
• BFCL May 14th Release (GPT-4o and Gemini) by @Fanjia-Yan in https://github.com/ShishirPatil/gorilla/pull/426
• [BFCL] update tree_sitter version in requirements.txt by @justinwangx in https://github.com/ShishirPatil/gorilla/pull/433
• Fix indentation in leaderboard README by @polm-stability in https://github.com/ShishirPatil/gorilla/pull/449
• Fix breaking changes due to updated Anthropic SDK by @eitanturok in https://github.com/ShishirPatil/gorilla/pull/452

New Contributors

• @zuxin666 made their first contribution in https://github.com/ShishirPatil/gorilla/pull/335
• @JasonZhu1313 made their first contribution in https://github.com/ShishirPatil/gorilla/pull/364
• @ricklamers made their first contribution in https://github.com/ShishirPatil/gorilla/pull/367
• @eitanturok made their first contribution in https://github.com/ShishirPatil/gorilla/pull/369
• @harry-cohere made their first contribution in https://github.com/ShishirPatil/gorilla/pull/350
• @cedricvidal made their first contribution in https://github.com/ShishirPatil/gorilla/pull/381
• @aastroza made their first contribution in https://github.com/ShishirPatil/gorilla/pull/405
• @vandyxiaowei made their first contribution in https://github.com/ShishirPatil/gorilla/pull/406
• @justinwangx made their first contribution in https://github.com/ShishirPatil/gorilla/pull/433
• @polm-stability made their first contribution in https://github.com/ShishirPatil/gorilla/pull/449

Full Changelog: https://github.com/ShishirPatil/gorilla/compare/v0.2...v0.3