diff --git a/README.md b/README.md index 4981b26..ae7c2e1 100644 --- a/README.md +++ b/README.md @@ -17,14 +17,16 @@ tags: # Qwen2.5-Coder-1.5B-Instruct + + Chat + ## Introduction -Qwen2.5-Coder is the latest series of Code-Specific Qwen large language models (formerly known as CodeQwen). For Qwen2.5-Coder, we release three base language models and instruction-tuned language models, 1.5, 7 and 32 (coming soon) billion parameters. Qwen2.5-Coder brings the following improvements upon CodeQwen1.5: +Qwen2.5-Coder is the latest series of Code-Specific Qwen large language models (formerly known as CodeQwen). As of now, Qwen2.5-Coder has covered six mainstream model sizes, 0.5, 1.5, 3, 7, 14, 32 billion parameters, to meet the needs of different developers. Qwen2.5-Coder brings the following improvements upon CodeQwen1.5: -- Significantly improvements in **code generation**, **code reasoning** and **code fixing**. Base on the strong Qwen2.5, we scale up the training tokens into 5.5 trillion including source code, text-code grounding, Synthetic data, etc. +- Significantly improvements in **code generation**, **code reasoning** and **code fixing**. Base on the strong Qwen2.5, we scale up the training tokens into 5.5 trillion including source code, text-code grounding, Synthetic data, etc. Qwen2.5-Coder-32B has become the current state-of-the-art open-source codeLLM, with its coding abilities matching those of GPT-4o. - A more comprehensive foundation for real-world applications such as **Code Agents**. Not only enhancing coding capabilities but also maintaining its strengths in mathematics and general competencies. -- **Long-context Support** up to 128K tokens. **This repo contains the instruction-tuned 1.5B Qwen2.5-Coder model**, which has the following features: - Type: Causal Language Models @@ -34,10 +36,9 @@ Qwen2.5-Coder is the latest series of Code-Specific Qwen large language models ( - Number of Paramaters (Non-Embedding): 1.31B - Number of Layers: 28 - Number of Attention Heads (GQA): 12 for Q and 2 for KV -- Context Length: Full 131,072 tokens - - Please refer to [this section](#processing-long-texts) for detailed instructions on how to deploy Qwen2.5 for handling long texts. +- Context Length: Full 32,768 tokens -For more details, please refer to our [blog](https://qwenlm.github.io/blog/qwen2.5-coder/), [GitHub](https://github.com/QwenLM/Qwen2.5-Coder), and [Documentation](https://qwen.readthedocs.io/en/latest/). +For more details, please refer to our [blog](https://qwenlm.github.io/blog/qwen2.5-coder-family/), [GitHub](https://github.com/QwenLM/Qwen2.5-Coder), [Documentation](https://qwen.readthedocs.io/en/latest/), [Arxiv](https://arxiv.org/abs/2409.12186). ## Requirements @@ -53,9 +54,9 @@ KeyError: 'qwen2' Here provides a code snippet with `apply_chat_template` to show you how to load the tokenizer and model and how to generate contents. ```python -from modelscope import AutoModelForCausalLM, AutoTokenizer +from transformers import AutoModelForCausalLM, AutoTokenizer -model_name = "qwen/Qwen2.5-Coder-1.5B-Instruct" +model_name = "Qwen/Qwen2.5-Coder-1.5B-Instruct" model = AutoModelForCausalLM.from_pretrained( model_name, @@ -87,31 +88,10 @@ generated_ids = [ response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0] ``` -### Processing Long Texts - -The current `config.json` is set for context length up to 32,768 tokens. -To handle extensive inputs exceeding 32,768 tokens, we utilize [YaRN](https://arxiv.org/abs/2309.00071), a technique for enhancing model length extrapolation, ensuring optimal performance on lengthy texts. - -For supported frameworks, you could add the following to `config.json` to enable YaRN: -```json -{ - ..., - "rope_scaling": { - "factor": 4.0, - "original_max_position_embeddings": 32768, - "type": "yarn" - } -} -``` - -For deployment, we recommend using vLLM. -Please refer to our [Documentation](https://qwen.readthedocs.io/en/latest/deployment/vllm.html) for usage if you are not familar with vLLM. -Presently, vLLM only supports static YARN, which means the scaling factor remains constant regardless of input length, **potentially impacting performance on shorter texts**. -We advise adding the `rope_scaling` configuration only when processing long contexts is required. ## Evaluation & Performance -Detailed evaluation results are reported in this [📑 blog](https://qwenlm.github.io/blog/qwen2.5-coder/). +Detailed evaluation results are reported in this [📑 blog](https://qwenlm.github.io/blog/qwen2.5-coder-family/). For requirements on GPU memory and the respective throughput, see results [here](https://qwen.readthedocs.io/en/latest/benchmark/speed_benchmark.html). @@ -120,12 +100,11 @@ For requirements on GPU memory and the respective throughput, see results [here] If you find our work helpful, feel free to give us a cite. ``` -@misc{qwen2.5, - title = {Qwen2.5: A Party of Foundation Models}, - url = {https://qwenlm.github.io/blog/qwen2.5/}, - author = {Qwen Team}, - month = {September}, - year = {2024} +@article{hui2024qwen2, + title={Qwen2. 5-Coder Technical Report}, + author={Hui, Binyuan and Yang, Jian and Cui, Zeyu and Yang, Jiaxi and Liu, Dayiheng and Zhang, Lei and Liu, Tianyu and Zhang, Jiajun and Yu, Bowen and Dang, Kai and others}, + journal={arXiv preprint arXiv:2409.12186}, + year={2024} } @article{qwen2, title={Qwen2 Technical Report},