open-llms - 🤖 A list of open LLMs available for commercial use.
Created at: 2023-05-06 00:35:37
Language:
License: Apache-2.0
Open LLMs
These LLMs are all licensed for commercial use (e.g., Apache 2.0, MIT, OpenRAIL-M). Contributions welcome!
Open LLMs for code
| Language Model | Release Date | Checkpoints | Paper/Blog | Params (B) | Context Length | Licence |
|---|---|---|---|---|---|---|
| SantaCoder | TODO | santacoder | SantaCoder: don't reach for the stars! | 1.1 | 2048 | OpenRAIL-M v1 |
| StarCoder | TODO | starcoder | StarCoder: A State-of-the-Art LLM for Code, StarCoder: May the source be with you! | 15 | 8192 | OpenRAIL-M v1 |
| StarChat Alpha | TODO | starchat-alpha | Creating a Coding Assistant with StarCoder | 16 | 8192 | OpenRAIL-M v1 |
| Replit Code | TODO | replit-code-v1-3b | Training a SOTA Code LLM in 1 week and Quantifying the Vibes — with Reza Shabani of Replit | 2.7 | infinity? (ALiBi) | CC BY-SA-4.0 |
| CodeGen2 | TODO | codegen2 1B-16B | CodeGen2: Lessons for Training LLMs on Programming and Natural Languages | 1 - 16 | 2048 | Apache 2.0 |
Open LLM datasets for pre-training
| Name | Release Date | Paper/Blog | Dataset | Tokens (T) | License |
|---|---|---|---|---|---|
| starcoderdata | 2023/05 | StarCoder: A State-of-the-Art LLM for Code | starcoderdata | 0.25 | Apache 2.0 |
| RedPajama | 2023/04 | RedPajama, a project to create leading open-source models, starts by reproducing LLaMA training dataset of over 1.2 trillion tokens | RedPajama-Data | 1.2 | Apache 2.0 |
Open LLM datasets for instruction-tuning
| Name | Release Date | Paper/Blog | Dataset | Samples (K) | License |
|---|---|---|---|---|---|
| MPT-7B-Instruct | 2023/05 | Introducing MPT-7B: A New Standard for Open-Source, Commercially Usable LLMs | dolly_hhrlhf | 59 | CC BY-SA-3.0 |
| databricks-dolly-15k | 2023/04 | Free Dolly: Introducing the World's First Truly Open Instruction-Tuned LLM | databricks-dolly-15k | 15 | CC BY-SA-3.0 |
| OIG (Open Instruction Generalist) | 2023/03 | THE OIG DATASET | OIG | 44,000 | Apache 2.0 |
Open LLM datasets for alignment-tuning
| Name | Release Date | Paper/Blog | Dataset | Samples (K) | License |
|---|---|---|---|---|---|
| OpenAssistant Conversations Dataset | 2023/04 | OpenAssistant Conversations - Democratizing Large Language Model Alignment | oasst1 | 161 | Apache 2.0 |
Evals on open LLMs
- Leaderboard by lmsys.org
- Evals by MosaicML
- Holistic Evaluation of Language Models (HELM)
- LLM-Leaderboard
- TextSynth Server Benchmarks
What do the licences mean?
- Apache 2.0: Allows users to use the software for any purpose, to distribute it, to modify it, and to distribute modified versions of the software under the terms of the license, without concern for royalties.
- MIT: Similar to Apache 2.0 but shorter and simpler. Also, in contrast to Apache 2.0, does not require stating any significant changes to the original code.
- CC BY-SA-4.0: Allows (i) copying and redistributing the material and (ii) remixing, transforming, and building upon the material for any purpose, even commercially. But if you do the latter, you must distribute your contributions under the same license as the original. (Thus, may not be viable for internal teams.)
- OpenRAIL-M v1: Allows royalty-free access and flexible downstream use and sharing of the model and modifications of it, and comes with a set of use restrictions (see Attachment A)
Disclaimer: The information provided in this repo does not, and is not intended to, constitute legal advice. Maintainers of this repo are not responsible for the actions of third parties who use the models. Please consult an attorney before using models for commercial purposes.
Improvements
- [x] Complete entries for context length, and check entries with
? - [ ]
Add number of tokens trained?(see considerations) - [ ] Add (links to) training code?
- [ ] Add (links to) eval benchmarks?
热门帖子
热门github
1
2
4
The Python Risk Identification Tool for generative AI (PyRIT) is an open access automation framework to empower security professionals and machine learning engineers to proactively find risks in their generative AI systems.
(翻译：用于生成式 AI 的 Python 风险识别工具 (PyRIT) 是一个开放式访问自动化框架，使安全专业人员和机器学习工程师能够主动发现其生成式 AI 系统中的风险。)
5
7
8
9
Mamba is a new state space model architecture showing promising performance on information-dense data such as language modeling, where previous subquadratic models fall short of Transformers. It is based on the line of progress on structured state space models, with an efficient hardware-aware design and implementation in the spirit of FlashAttention.
(翻译：Mamba 是一种新的状态空间模型架构，在信息密集型数据（例如语言建模）上显示出良好的性能，而之前的二次模型在 Transformers 方面存在不足。它基于结构化状态空间模型的进展，并本着FlashAttention的精神进行高效的硬件感知设计和实现。)
10
11
13