Chinese-alpaca-lora - 骆驼:A Chinese finetuned instruction LLaMA. Developed by 陈启源 @ 华中师范大学 & 李鲁鲁 @ 商汤科技 & 冷子昂 @ 商汤科技

Created at: 2023-03-21 10:43:12
Language: Jupyter Notebook
License: Apache-2.0

骆驼(Luotuo): Chinese-alpaca-lora

骆驼(Luotuo) is the Chinese pinyin(pronunciation) of camel

A Chinese finetuned instruction LLaMA. Developed by 冷子昂 @ 商汤科技, 陈启源 @ 华中师范大学(Third year undergraduate student) and 李鲁鲁 @ 商汤科技


This is NOT an official product of SenseTime

We named project in Luotuo(Camel) because both LLaMA and alpaca are all belongs to Artiodactyla-Camelidae(偶蹄目-骆驼科)


[2023-3-25] Luotuo-1.0 is in training! Thanks for all sponsors!

[2023-3-25] CamelBell(驼铃), tuning Chinese Data on Chinese based model GLM is now an individual repo. We may move original Luotuo into a new repo also.

[2023-3-24] We've just released CamelBell(驼铃): tuning Chinese LLM with very few data on GLM-6B via LoRA, try here Open In Colab , we may create a new repo soon

A Quick Start

Colab Link detail
CamelBell quick evaluation Open In Colab Tuoling specific Evaluation Code
A quick evaluation Open In Colab Evaluation code with standard HuggingFace pipeline
Bot with Interface Open In Colab Interactive Chatting Bot using Gradio
Training Code To be released Training code, run on colab
Data Translation Open In Colab Translation alpaca.json into Chinese

Trained Model

Model Name Training Data and Setting
luotuo-lora-7b-0.1 Trained on Translated alpaca 52k data
luotuo-lora-7b-0.3 Trained on Translated alpaca 52k data + guanaco, 1 epoch
luotuo-lora-7b-0.9 (In Plan) cleaned alpaca 52k + full guanaco

luotuo-lora-7b-0.3 the performance seems have significant improvement than 0.1, even just after 1 epoch.

But 1 epoch cost us 7 hours to train, which cost more than $10. We will keep continue tuning it later.

Qualitative Result(0.3)

Evaluation code are now in notebook/evaluation_code.ipynb

Good Examples

Input: 中国的首都在哪里?
Luotuo-Output: 中国的首都是北京。
Input: 麦当劳是哪个国家的?
Luotuo-Output: 美国。
Input: 2017年美国总统是谁
Luotuo-Output: 2017年美国总统是特朗普
Input: 华中师范大学在哪里
Luotuo-0.1-Output: 华中师范大学位于北京
Luotuo-0.3-Output: 华中师范大学在武汉市。

this example failed in 0.1

Bad Examples

Input: 商汤科技是做什么的?
Luotuo-0.1-Output: 商汤科技是一种技术,它用于创建互联网应用程序和服务。
Luotuo-0.3-Output: 商汤科技是一种技术,它可以用于创建新的产品和服务,以增加产品和服务的吸引力。它可以用于创建新的产品和服务,以增加产品和服务的吸引力。它可以用于创建新的产品和服务,以增加产品和服务的吸引力。它可以用于创建新的产品和服务,以增加产品和服务的吸引力。它可以用于创建新的产品和服务,以增加产品和服务的吸引力。它可以用于创建新的产品和服务,以增加产品和服务的吸引力。它可以用于创建新的
Input: 猫和狗的关系是什么
Luotuo-Output: 猫和狗之间的关系是友好的。它们通常在一起休息或玩耍。猫和狗之间的关系通常很好,它们通常在一起休息或玩耍。猫和狗之间的关系通常很好,它们通常在一起休息或玩耍。猫和狗之间的关系通常很好,它们通常在一起休息或玩耍。猫和狗之间的关系通常很好,它们通常在一起休息或玩耍。猫和狗之间的关系通常很好,它们通常在一起休息或玩耍。猫和狗之间的关系通常


We have tuned a Chinese LLaMA model baed on LLaMA, Stanford Alpaca, Alpaca LoRA, cabrita, Japanese-Alpaca-LoRA

The training code in in cleaning, if you are in very hurry, check the Japanese project and simply change the json training data file name.


This is an inbuilding project

The training code only made a slightly change on the Japanese-Alpaca-LoRA

A. 0.1 version model was trained on translated data, which translate the alpaca_data.json to Chinese using ChatGPT API. We paid around US $30-45 to translate the full dataset to chinese. Translated data is available. (trans_chinese_alpaca_data.json)

B. We are also plan to consider the data in Guanaco hikariming's alpaca_chinese_dataset and carbonz0‘s alpaca-chinese-dataset, may updated it into later version.

We plan to upload two different models A and B, because the provider of B claim the clean data will bring significant improvement.


Top 3 Sponsors

Time Sponsor Amount
2023/3/25 肖** 520
2023/3/24 yiplee 512
2023/3/24 Hijun 500
2023/3/24 倪** 500

balance = 2374 now. Detailed balance see in




项目的资金流向将被公开,所有的资金将被用于数据的标注,训练算力的购买或者后续周边产品的发放。数据和算力的捐献也会一同总结在sponsorship的表格中。备用链接 二维码 , 支付宝账号

This was originally an exercise project for us, and we originally planned to train until version 1.0. However, the enthusiasm of the community exceeded our expectations. If you are willing to sponsor our project, you can scan this QR code and add this Alipay account, leaving your name.

All funds will be used for data annotation, purchase of training computing power, or distribution of subsequent peripheral products.

TODO and Be a Contributor

It seems that there are many follow-up tasks to be done after the basic version is completed. Many developers in the community have put forward more friendly suggestions, and I have put a longer TODO list in

inbuilding project

  • [X] translate alpaca json data into Chinese
  • [X] finetuning with lora(model 0.1)
  • [X] release 0.1 model (model A)
  • [X] model to hugging face, GUI demo
  • [X] train lora with more alpaca data(model 0.3)
  • [ ] (In Processing) train lora with more alpaca data(model 0.9)

We plan to use this Luotuo project as the git repository for the entire Chinese LLM project. After the completion of the original Luotuo: LLaMA-LoRA, it will be migrated to Luotuo-vanilla. The CamelBell, Loulan, Silk-Road and other derivative Chinese language model projects will gradually be added to the Luotuo project.


Please cite the repo if you use the data or code in this repo.

  author={Ziang Leng, Qiyuan Chen and Cheng Li},
  title = {Luotuo: An Instruction-following Chinese Language, LoRA tuning on LLaMA model},
  year = {2023},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{}},