Ollama私有化部署大语言模型LLM

目录

一、Ollama介绍

二、安装Ollama

  1、标准安装

  2、国内加速

三、升级Ollama版本

四、使用Ollama

  1、启动ollama服务

systemctl start ollama.service 

ollama serve  

  2、使用ollama命令

ollama run 运行模型

ollama ps 查看正在运行的模型

ollama list 查看(本地)已保存的模型

ollama show 查看(本地)已保存模型简要信息

 五、通过http访问Ollama

 1、列出本地已有的模型

 2、列出本正在运行的模型

 3、获取指定模型详情

 4、模型推理（生成内容）

generate接口

chat接口

 5、可复现回答

 6、覆盖默认的system指令

 7、携带上下文

 8、删除模型

五、查看Ollama服务日志

附、Ollama拉取Huggingface上的模型

---

一、Ollama介绍

  Ollama（官方网址：https://ollama.com/）是一个开源的大型语言模型（LLM）服务工具，旨在简化在本地环境中部署和运行这些模型的过程。它支持多种操作系统，包括Windows、macOS和Linux，并且可以通过Docker容器进行管理。Ollama封装了llama.cpp库，并提供与OpenAI兼容的API，支持多种语言模型如Llama 3、Mistral和Gemma。此外，Ollama还支持并行请求和多线程操作，提高了效率。用户可以通过简单的命令行工具或Web界面（Open WebUI、Hollama等）来管理和运行模型。Ollama采用了docker技术以实现简化使用，所以使用ollama命令风格类似docker。

  同时，Ollama提供了官方的模型registry中心（网站：https://www.ollama.com/models），上面是官方提供的模型列表（其地位类似于docker的DockerHub）。

  Ollama的特点：

• 本地推理：

   有无GPU均可：即使无GPU，仅CPU也能让开源模型（如 LLaMA、GPT-Neo、Mistral 等）跑起来。还可GPU+CPU混合：若自动检测到机器上存在GPU（NVIDIA或者AMD），自动同时GPU+CPU，充分发挥两者的能力。

• 轻量易用：

安装方式简洁，一键下载安装。只需一个命令行工具就能在本地完成私有化部署，一键就能自动下载运行模型并进行对话、推理。

• 量化优化：

支持对常见大语言模型做 4-bit 或 8-bit 等量化Quantization，通过量化进一步降低硬件资源占用。

• 发展活跃：

在 GitHub 上有不错的社区支持和更新节奏，适合初中级开发者快速上手。

   

二、安装Ollama

  1、标准安装

   本文以Linux为例，如下图官网，通过一行命令一键完成Linux上完成安装。该句命令的作用是：通过curl从网上获取install.sh并输出至stdout（不将该install.sh保存至本地文件），然后通过sh执行。

除了一键安装，还可以通过下方的“View script source”查看install.sh的内容。可以通过“Manual install instructions”将install.sh中的各步骤通过手工一步步执行，而不是一键完成安装。

   注意：需具有sudo权限的普通用户或者root用户执行安装命令。

   安装目标目录的基目录为“/usr/local/” 或 “/usr/” 或 “/” 目录下（具体看安装时提示），在基目录下的：

• bin/ollama文件，为可执行文件
• lib/ollama子目录，为所需的lib库。

  一键安装会自动：创建一个ollama用户，还会自动纳入Linux服务管理以随Linux启动而启动；若存在Nvidia GPU硬件则还会安装CUDA等。

  若在安装Ollama过程中报“systemctl相关错误”， 或者在执行systemctl命令时报“System has not been booted with systemd as init system (PID 1). Can't operate.Failed to connect to bus: Host is down”，则需要先在Linux安装system和systemctl，以Ubuntu为例安装命令如下：

apt-get install systemd -y

apt-get install systemctl -y

  2、国内加速

      在标准安装的install.sh中，会自动根据当前Linux平台通过“https://ollama.com/download/文件名”下载安装所需的tgz文件（该URL实际最终会通过“https://github.com/ollama/ollama/releases/latest/download/文件名”下载），所需下载的tgz文件大约1~2GB。

     然而国内访问github速度通常几百KB/s比较慢，可以通过文件加速器网站下载。下面以加速器网站“https://github.xzc888.top”为例加速（也可使用其他加速器网站），使用该加速器网站的方法是将URL中的“github.com”替换为“github.xzc888.top”。具体安装ollama过程如下：

#将脚本下载，并保存至本地。保存的文件名为“install_ollama.sh”
curl -fsSL https://ollama.com/install.sh -o install_ollama.sh#将该文件中的下载URL前缀替换为加速后的URL前缀
sed -i 's#https://ollama.com/download/#https://github.xzc888.top/ollama/ollama/releases/latest/download/#g' install_ollama.sh#执行ollama安装
sh install_ollama.sh#最后删除临时sh脚本
rm install_ollama.sh

三、升级Ollama版本

   若Linux中已经安装ollama，希望升级新版ollama软件，怎么办？

   只需再次执行以上安装步骤即可使用新版覆盖已有的老版。

四、使用Ollama

   类似docker，安装完ollama后需要先启动ollama服务端，然后才能通过命令操作。

  1、启动ollama服务

   启动ollama服务(服务端)可以通过：

• systemctl start ollama.service 

     通过Linux服务启动，该启动将会使用ollama用户运行进程。该方式启动默认使用/usr/share/ollama/.ollama/models目录作为存放模型的目录。

      另：ollama服务通过OLLAMA_HOST环境确定服务侦听，该环境变量默认为127.0.0.1:11434，所以只能供本机访问。为了让其他机器访问也能访问ollama服务。编辑/etc/systemd/system/ollama.service文件，对Environment增加设置OLLAMA_HOST环境变量，如下

    然后执行：systemctl daemon-reload; systemctl restart ollama.service 使其生效。

• ollama serve  

     直接通过ollama命令启动，该启动方式默认使用启动命令用户的~/.ollama/models目录作为存放模型的目录。

    附：ollama服务通过OLLAMA_HOST环境确定服务侦听，该环境变量默认为127.0.0.1:11434，所以只能供本机访问。为了让其他机器访问也能访问ollama服务。先在Linux中设置环境变量export OLLAMA_HOST=0.0.0.0:11434，然后再启动ollama服务。

  2、使用ollama命令

     除了ollama serve命令为服务端命令外，其他命令为ollama客户端命令。如下为客户端命令（每个命令有对应的环境变量，具体通过命令加--help查看）。

     客户端命令通过OLLAMA_HOST环境变量指定ollama服务所在的IP和端口，该环境变量默认为127.0.0.1:11434，可在使用时修改环境变量。

•   create      Create a model from a Modelfile
•   show        Show information for a model
•   run         Run a model
•   stop        Stop a running model
•   pull        Pull a model from a registry
•   push        Push a model to a registry
•   list        List models
•   ps          List running models
•   cp          Copy a model
•   rm          Remove a model

ollama run 运行模型

使用以下命令运行模型（若本地没有该模型，则自动从ollama官网获取）。

注意：这里的运行应当更准确理解为：加载模型至内存或显存。因为只有Ollama服务才涉及启动运行，模型不涉及启动运行，模型只是被Ollama服务加载，加载后即可由Ollama服务对外提供。

#运行模型（若本地没有该模型，则自动从ollama官网获取）
ollama run qwen2.5-coder
#或者增加--verbose，显示应答耗时、消耗token、token速度等信息
ollama run --verbose qwen2.5-coder

运行后，自动进入输入prompt对话框。如下：

还可以通过斜杠命令，查看模型信息、查看预设system提示词、查看预设提示词模版等等等。如下：

   其中/bye只是结束当前对话，不代表模型退出加载。

ollama ps 查看正在运行的模型

 注意，输出内容中：

• PROCESSOR 不是指消耗CPU比例。100% CPU表示模型全部加载至内存；100% GPU表示模型全部加载至CPU显存；48%/52% CPU/GPU表示模型分别按比例加载至内存和显存。这个Ollama使用的词不准确，应该用MEM和GMEM就比较准确。
• UNTIL 表示模型还剩多长时间就退出加载，既还剩多长时间从内存或显存中卸载该模型。时长可通过OLLAMA_KEEP_ALIVE环境变量，或者通过API接口的keep_alive参数动态改变。具体见官网说明：https://github.com/ollama/ollama/blob/main/docs/faq.md

ollama list 查看(本地)已保存的模型

ollama show 查看(本地)已保存模型简要信息

  包括模型基本信息、system提示词、License等

 五、通过http访问Ollama

   ollama服务端提供多个http接口供访问，通过http访问服务端使用各个大模型功能。注意提供http的不是大模型，而是ollama服务端（默认的11434端口是ollama服务的端口，不是模型的端口，模型也不存在端口）。所以通过http不仅能选择使用大模型，而且还可以创建、查看、删除模型

    下面以curl工具进行演示。

 1、列出本地已有的模型

curl http://localhost:11434/api/tags

 返回结果（已经格式化排版）如下：

{"models": [{"name": "qwen2.5-coder:latest","model": "qwen2.5-coder:latest","modified_at": "2025-01-09T08:43:12.893109668+08:00","size": 4683087519,"digest": "2b0496514337a3d5901f1d253d01726c890b721e891335a56d6e08cedf3e2cb0","details": {"parent_model": "","format": "gguf","family": "qwen2","families": ["qwen2"],"parameter_size": "7.6B","quantization_level": "Q4_K_M"}}, {"name": "qwen2.5-coder:0.5b","model": "qwen2.5-coder:0.5b","modified_at": "2025-01-08T19:32:09.640283014+08:00","size": 531081760,"digest": "d392ed348d5bb7847eaefe3fcb18e5bcc6433aecb68bf61b6e028d87292ab54f","details": {"parent_model": "","format": "gguf","family": "qwen2","families": ["qwen2"],"parameter_size": "494.03M","quantization_level": "Q8_0"}}]
}

 2、列出本正在运行的模型

curl http://localhost:11434/api/ps

 返回结果（已经格式化排版）如下：

{"models": [{"name": "qwen2.5-coder:0.5b","model": "qwen2.5-coder:0.5b","size": 958830592,"digest": "d392ed348d5bb7847eaefe3fcb18e5bcc6433aecb68bf61b6e028d87292ab54f","details": {"parent_model": "","format": "gguf","family": "qwen2","families": ["qwen2"],"parameter_size": "494.03M","quantization_level": "Q8_0"},"expires_at": "2025-01-09T14:31:47.581505612+08:00","size_vram": 0}]
}

 3、获取指定模型详情

curl -X POST http://localhost:11434/api/show -d '{"name": "qwen2.5-coder:0.5b","verbose": false 
}'

 返回结果（已经格式化排版）如下（如果开启verbose信息会更多）

{"license": "\n                                 Apache License\n                           Version 2.0, January 2004\n                        http://www.apache.org/licenses/\n\n   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION\n\n   1. Definitions.\n\n      \"License\" shall mean the terms and conditions for use, reproduction,\n      and distribution as defined by Sections 1 through 9 of this document.\n\n      \"Licensor\" shall mean the copyright owner or entity authorized by\n      the copyright owner that is granting the License.\n\n      \"Legal Entity\" shall mean the union of the acting entity and all\n      other entities that control, are controlled by, or are under common\n      control with that entity. For the purposes of this definition,\n      \"control\" means (i) the power, direct or indirect, to cause the\n      direction or management of such entity, whether by contract or\n      otherwise, or (ii) ownership of fifty percent (50%) or more of the\n      outstanding shares, or (iii) beneficial ownership of such entity.\n\n      \"You\" (or \"Your\") shall mean an individual or Legal Entity\n      exercising permissions granted by this License.\n\n      \"Source\" form shall mean the preferred form for making modifications,\n      including but not limited to software source code, documentation\n      source, and configuration files.\n\n      \"Object\" form shall mean any form resulting from mechanical\n      transformation or translation of a Source form, including but\n      not limited to compiled object code, generated documentation,\n      and conversions to other media types.\n\n      \"Work\" shall mean the work of authorship, whether in Source or\n      Object form, made available under the License, as indicated by a\n      copyright notice that is included in or attached to the work\n      (an example is provided in the Appendix below).\n\n      \"Derivative Works\" shall mean any work, whether in Source or Object\n      form, that is based on (or derived from) the Work and for which the\n      editorial revisions, annotations, elaborations, or other modifications\n      represent, as a whole, an original work of authorship. For the purposes\n      of this License, Derivative Works shall not include works that remain\n      separable from, or merely link (or bind by name) to the interfaces of,\n      the Work and Derivative Works thereof.\n\n      \"Contribution\" shall mean any work of authorship, including\n      the original version of the Work and any modifications or additions\n      to that Work or Derivative Works thereof, that is intentionally\n      submitted to Licensor for inclusion in the Work by the copyright owner\n      or by an individual or Legal Entity authorized to submit on behalf of\n      the copyright owner. For the purposes of this definition, \"submitted\"\n      means any form of electronic, verbal, or written communication sent\n      to the Licensor or its representatives, including but not limited to\n      communication on electronic mailing lists, source code control systems,\n      and issue tracking systems that are managed by, or on behalf of, the\n      Licensor for the purpose of discussing and improving the Work, but\n      excluding communication that is conspicuously marked or otherwise\n      designated in writing by the copyright owner as \"Not a Contribution.\"\n\n      \"Contributor\" shall mean Licensor and any individual or Legal Entity\n      on behalf of whom a Contribution has been received by Licensor and\n      subsequently incorporated within the Work.\n\n   2. Grant of Copyright License. Subject to the terms and conditions of\n      this License, each Contributor hereby grants to You a perpetual,\n      worldwide, non-exclusive, no-charge, royalty-free, irrevocable\n      copyright license to reproduce, prepare Derivative Works of,\n      publicly display, publicly perform, sublicense, and distribute the\n      Work and such Derivative Works in Source or Object form.\n\n   3. Grant of Patent License. Subject to the terms and conditions of\n      this License, each Contributor hereby grants to You a perpetual,\n      worldwide, non-exclusive, no-charge, royalty-free, irrevocable\n      (except as stated in this section) patent license to make, have made,\n      use, offer to sell, sell, import, and otherwise transfer the Work,\n      where such license applies only to those patent claims licensable\n      by such Contributor that are necessarily infringed by their\n      Contribution(s) alone or by combination of their Contribution(s)\n      with the Work to which such Contribution(s) was submitted. If You\n      institute patent litigation against any entity (including a\n      cross-claim or counterclaim in a lawsuit) alleging that the Work\n      or a Contribution incorporated within the Work constitutes direct\n      or contributory patent infringement, then any patent licenses\n      granted to You under this License for that Work shall terminate\n      as of the date such litigation is filed.\n\n   4. Redistribution. You may reproduce and distribute copies of the\n      Work or Derivative Works thereof in any medium, with or without\n      modifications, and in Source or Object form, provided that You\n      meet the following conditions:\n\n      (a) You must give any other recipients of the Work or\n          Derivative Works a copy of this License; and\n\n      (b) You must cause any modified files to carry prominent notices\n          stating that You changed the files; and\n\n      (c) You must retain, in the Source form of any Derivative Works\n          that You distribute, all copyright, patent, trademark, and\n          attribution notices from the Source form of the Work,\n          excluding those notices that do not pertain to any part of\n          the Derivative Works; and\n\n      (d) If the Work includes a \"NOTICE\" text file as part of its\n          distribution, then any Derivative Works that You distribute must\n          include a readable copy of the attribution notices contained\n          within such NOTICE file, excluding those notices that do not\n          pertain to any part of the Derivative Works, in at least one\n          of the following places: within a NOTICE text file distributed\n          as part of the Derivative Works; within the Source form or\n          documentation, if provided along with the Derivative Works; or,\n          within a display generated by the Derivative Works, if and\n          wherever such third-party notices normally appear. The contents\n          of the NOTICE file are for informational purposes only and\n          do not modify the License. You may add Your own attribution\n          notices within Derivative Works that You distribute, alongside\n          or as an addendum to the NOTICE text from the Work, provided\n          that such additional attribution notices cannot be construed\n          as modifying the License.\n\n      You may add Your own copyright statement to Your modifications and\n      may provide additional or different license terms and conditions\n      for use, reproduction, or distribution of Your modifications, or\n      for any such Derivative Works as a whole, provided Your use,\n      reproduction, and distribution of the Work otherwise complies with\n      the conditions stated in this License.\n\n   5. Submission of Contributions. Unless You explicitly state otherwise,\n      any Contribution intentionally submitted for inclusion in the Work\n      by You to the Licensor shall be under the terms and conditions of\n      this License, without any additional terms or conditions.\n      Notwithstanding the above, nothing herein shall supersede or modify\n      the terms of any separate license agreement you may have executed\n      with Licensor regarding such Contributions.\n\n   6. Trademarks. This License does not grant permission to use the trade\n      names, trademarks, service marks, or product names of the Licensor,\n      except as required for reasonable and customary use in describing the\n      origin of the Work and reproducing the content of the NOTICE file.\n\n   7. Disclaimer of Warranty. Unless required by applicable law or\n      agreed to in writing, Licensor provides the Work (and each\n      Contributor provides its Contributions) on an \"AS IS\" BASIS,\n      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or\n      implied, including, without limitation, any warranties or conditions\n      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A\n      PARTICULAR PURPOSE. You are solely responsible for determining the\n      appropriateness of using or redistributing the Work and assume any\n      risks associated with Your exercise of permissions under this License.\n\n   8. Limitation of Liability. In no event and under no legal theory,\n      whether in tort (including negligence), contract, or otherwise,\n      unless required by applicable law (such as deliberate and grossly\n      negligent acts) or agreed to in writing, shall any Contributor be\n      liable to You for damages, including any direct, indirect, special,\n      incidental, or consequential damages of any character arising as a\n      result of this License or out of the use or inability to use the\n      Work (including but not limited to damages for loss of goodwill,\n      work stoppage, computer failure or malfunction, or any and all\n      other commercial damages or losses), even if such Contributor\n      has been advised of the possibility of such damages.\n\n   9. Accepting Warranty or Additional Liability. While redistributing\n      the Work or Derivative Works thereof, You may choose to offer,\n      and charge a fee for, acceptance of support, warranty, indemnity,\n      or other liability obligations and/or rights consistent with this\n      License. However, in accepting such obligations, You may act only\n      on Your own behalf and on Your sole responsibility, not on behalf\n      of any other Contributor, and only if You agree to indemnify,\n      defend, and hold each Contributor harmless for any liability\n      incurred by, or claims asserted against, such Contributor by reason\n      of your accepting any such warranty or additional liability.\n\n   END OF TERMS AND CONDITIONS\n\n   APPENDIX: How to apply the Apache License to your work.\n\n      To apply the Apache License to your work, attach the following\n      boilerplate notice, with the fields enclosed by brackets \"[]\"\n      replaced with your own identifying information. (Don't include\n      the brackets!)  The text should be enclosed in the appropriate\n      comment syntax for the file format. We also recommend that a\n      file or class name and description of purpose be included on the\n      same \"printed page\" as the copyright notice for easier\n      identification within third-party archives.\n\n   Copyright 2024 Alibaba Cloud\n\n   Licensed under the Apache License, Version 2.0 (the \"License\");\n   you may not use this file except in compliance with the License.\n   You may obtain a copy of the License at\n\n       http://www.apache.org/licenses/LICENSE-2.0\n\n   Unless required by applicable law or agreed to in writing, software\n   distributed under the License is distributed on an \"AS IS\" BASIS,\n   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n   See the License for the specific language governing permissions and\n   limitations under the License.","modelfile": "# Modelfile generated by \"ollama show\"\n# To build a new Modelfile based on this, replace FROM with:\n# FROM qwen2.5-coder:0.5b\n\nFROM /usr/share/ollama/.ollama/models/blobs/sha256-828125e28bf46a219fa4f75b6982cb0c41fd9187467abe91c9b175287945b7ef\nTEMPLATE \"\"\"{{- if .Suffix }}\u003c|fim_prefix|\u003e{{ .Prompt }}\u003c|fim_suffix|\u003e{{ .Suffix }}\u003c|fim_middle|\u003e\n{{- else if .Messages }}\n{{- if or .System .Tools }}\u003c|im_start|\u003esystem\n{{- if .System }}\n{{ .System }}\n{{- end }}\n{{- if .Tools }}\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within \u003ctools\u003e\u003c/tools\u003e XML tags:\n\u003ctools\u003e\n{{- range .Tools }}\n{\"type\": \"function\", \"function\": {{ .Function }}}\n{{- end }}\n\u003c/tools\u003e\n\nFor each function call, return a json object with function name and arguments within \u003ctool_call\u003e\u003c/tool_call\u003e XML tags:\n\u003ctool_call\u003e\n{\"name\": \u003cfunction-name\u003e, \"arguments\": \u003cargs-json-object\u003e}\n\u003c/tool_call\u003e\n{{- end }}\u003c|im_end|\u003e\n{{ end }}\n{{- range $i, $_ := .Messages }}\n{{- $last := eq (len (slice $.Messages $i)) 1 -}}\n{{- if eq .Role \"user\" }}\u003c|im_start|\u003euser\n{{ .Content }}\u003c|im_end|\u003e\n{{ else if eq .Role \"assistant\" }}\u003c|im_start|\u003eassistant\n{{ if .Content }}{{ .Content }}\n{{- else if .ToolCalls }}\u003ctool_call\u003e\n{{ range .ToolCalls }}{\"name\": \"{{ .Function.Name }}\", \"arguments\": {{ .Function.Arguments }}}\n{{ end }}\u003c/tool_call\u003e\n{{- end }}{{ if not $last }}\u003c|im_end|\u003e\n{{ end }}\n{{- else if eq .Role \"tool\" }}\u003c|im_start|\u003euser\n\u003ctool_response\u003e\n{{ .Content }}\n\u003c/tool_response\u003e\u003c|im_end|\u003e\n{{ end }}\n{{- if and (ne .Role \"assistant\") $last }}\u003c|im_start|\u003eassistant\n{{ end }}\n{{- end }}\n{{- else }}\n{{- if .System }}\u003c|im_start|\u003esystem\n{{ .System }}\u003c|im_end|\u003e\n{{ end }}{{ if .Prompt }}\u003c|im_start|\u003euser\n{{ .Prompt }}\u003c|im_end|\u003e\n{{ end }}\u003c|im_start|\u003eassistant\n{{ end }}{{ .Response }}{{ if .Response }}\u003c|im_end|\u003e{{ end }}\"\"\"\nSYSTEM You are Qwen, created by Alibaba Cloud. You are a helpful assistant.\nLICENSE \"\"\"\n                                 Apache License\n                           Version 2.0, January 2004\n                        http://www.apache.org/licenses/\n\n   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION\n\n   1. Definitions.\n\n      \"License\" shall mean the terms and conditions for use, reproduction,\n      and distribution as defined by Sections 1 through 9 of this document.\n\n      \"Licensor\" shall mean the copyright owner or entity authorized by\n      the copyright owner that is granting the License.\n\n      \"Legal Entity\" shall mean the union of the acting entity and all\n      other entities that control, are controlled by, or are under common\n      control with that entity. For the purposes of this definition,\n      \"control\" means (i) the power, direct or indirect, to cause the\n      direction or management of such entity, whether by contract or\n      otherwise, or (ii) ownership of fifty percent (50%) or more of the\n      outstanding shares, or (iii) beneficial ownership of such entity.\n\n      \"You\" (or \"Your\") shall mean an individual or Legal Entity\n      exercising permissions granted by this License.\n\n      \"Source\" form shall mean the preferred form for making modifications,\n      including but not limited to software source code, documentation\n      source, and configuration files.\n\n      \"Object\" form shall mean any form resulting from mechanical\n      transformation or translation of a Source form, including but\n      not limited to compiled object code, generated documentation,\n      and conversions to other media types.\n\n      \"Work\" shall mean the work of authorship, whether in Source or\n      Object form, made available under the License, as indicated by a\n      copyright notice that is included in or attached to the work\n      (an example is provided in the Appendix below).\n\n      \"Derivative Works\" shall mean any work, whether in Source or Object\n      form, that is based on (or derived from) the Work and for which the\n      editorial revisions, annotations, elaborations, or other modifications\n      represent, as a whole, an original work of authorship. For the purposes\n      of this License, Derivative Works shall not include works that remain\n      separable from, or merely link (or bind by name) to the interfaces of,\n      the Work and Derivative Works thereof.\n\n      \"Contribution\" shall mean any work of authorship, including\n      the original version of the Work and any modifications or additions\n      to that Work or Derivative Works thereof, that is intentionally\n      submitted to Licensor for inclusion in the Work by the copyright owner\n      or by an individual or Legal Entity authorized to submit on behalf of\n      the copyright owner. For the purposes of this definition, \"submitted\"\n      means any form of electronic, verbal, or written communication sent\n      to the Licensor or its representatives, including but not limited to\n      communication on electronic mailing lists, source code control systems,\n      and issue tracking systems that are managed by, or on behalf of, the\n      Licensor for the purpose of discussing and improving the Work, but\n      excluding communication that is conspicuously marked or otherwise\n      designated in writing by the copyright owner as \"Not a Contribution.\"\n\n      \"Contributor\" shall mean Licensor and any individual or Legal Entity\n      on behalf of whom a Contribution has been received by Licensor and\n      subsequently incorporated within the Work.\n\n   2. Grant of Copyright License. Subject to the terms and conditions of\n      this License, each Contributor hereby grants to You a perpetual,\n      worldwide, non-exclusive, no-charge, royalty-free, irrevocable\n      copyright license to reproduce, prepare Derivative Works of,\n      publicly display, publicly perform, sublicense, and distribute the\n      Work and such Derivative Works in Source or Object form.\n\n   3. Grant of Patent License. Subject to the terms and conditions of\n      this License, each Contributor hereby grants to You a perpetual,\n      worldwide, non-exclusive, no-charge, royalty-free, irrevocable\n      (except as stated in this section) patent license to make, have made,\n      use, offer to sell, sell, import, and otherwise transfer the Work,\n      where such license applies only to those patent claims licensable\n      by such Contributor that are necessarily infringed by their\n      Contribution(s) alone or by combination of their Contribution(s)\n      with the Work to which such Contribution(s) was submitted. If You\n      institute patent litigation against any entity (including a\n      cross-claim or counterclaim in a lawsuit) alleging that the Work\n      or a Contribution incorporated within the Work constitutes direct\n      or contributory patent infringement, then any patent licenses\n      granted to You under this License for that Work shall terminate\n      as of the date such litigation is filed.\n\n   4. Redistribution. You may reproduce and distribute copies of the\n      Work or Derivative Works thereof in any medium, with or without\n      modifications, and in Source or Object form, provided that You\n      meet the following conditions:\n\n      (a) You must give any other recipients of the Work or\n          Derivative Works a copy of this License; and\n\n      (b) You must cause any modified files to carry prominent notices\n          stating that You changed the files; and\n\n      (c) You must retain, in the Source form of any Derivative Works\n          that You distribute, all copyright, patent, trademark, and\n          attribution notices from the Source form of the Work,\n          excluding those notices that do not pertain to any part of\n          the Derivative Works; and\n\n      (d) If the Work includes a \"NOTICE\" text file as part of its\n          distribution, then any Derivative Works that You distribute must\n          include a readable copy of the attribution notices contained\n          within such NOTICE file, excluding those notices that do not\n          pertain to any part of the Derivative Works, in at least one\n          of the following places: within a NOTICE text file distributed\n          as part of the Derivative Works; within the Source form or\n          documentation, if provided along with the Derivative Works; or,\n          within a display generated by the Derivative Works, if and\n          wherever such third-party notices normally appear. The contents\n          of the NOTICE file are for informational purposes only and\n          do not modify the License. You may add Your own attribution\n          notices within Derivative Works that You distribute, alongside\n          or as an addendum to the NOTICE text from the Work, provided\n          that such additional attribution notices cannot be construed\n          as modifying the License.\n\n      You may add Your own copyright statement to Your modifications and\n      may provide additional or different license terms and conditions\n      for use, reproduction, or distribution of Your modifications, or\n      for any such Derivative Works as a whole, provided Your use,\n      reproduction, and distribution of the Work otherwise complies with\n      the conditions stated in this License.\n\n   5. Submission of Contributions. Unless You explicitly state otherwise,\n      any Contribution intentionally submitted for inclusion in the Work\n      by You to the Licensor shall be under the terms and conditions of\n      this License, without any additional terms or conditions.\n      Notwithstanding the above, nothing herein shall supersede or modify\n      the terms of any separate license agreement you may have executed\n      with Licensor regarding such Contributions.\n\n   6. Trademarks. This License does not grant permission to use the trade\n      names, trademarks, service marks, or product names of the Licensor,\n      except as required for reasonable and customary use in describing the\n      origin of the Work and reproducing the content of the NOTICE file.\n\n   7. Disclaimer of Warranty. Unless required by applicable law or\n      agreed to in writing, Licensor provides the Work (and each\n      Contributor provides its Contributions) on an \"AS IS\" BASIS,\n      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or\n      implied, including, without limitation, any warranties or conditions\n      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A\n      PARTICULAR PURPOSE. You are solely responsible for determining the\n      appropriateness of using or redistributing the Work and assume any\n      risks associated with Your exercise of permissions under this License.\n\n   8. Limitation of Liability. In no event and under no legal theory,\n      whether in tort (including negligence), contract, or otherwise,\n      unless required by applicable law (such as deliberate and grossly\n      negligent acts) or agreed to in writing, shall any Contributor be\n      liable to You for damages, including any direct, indirect, special,\n      incidental, or consequential damages of any character arising as a\n      result of this License or out of the use or inability to use the\n      Work (including but not limited to damages for loss of goodwill,\n      work stoppage, computer failure or malfunction, or any and all\n      other commercial damages or losses), even if such Contributor\n      has been advised of the possibility of such damages.\n\n   9. Accepting Warranty or Additional Liability. While redistributing\n      the Work or Derivative Works thereof, You may choose to offer,\n      and charge a fee for, acceptance of support, warranty, indemnity,\n      or other liability obligations and/or rights consistent with this\n      License. However, in accepting such obligations, You may act only\n      on Your own behalf and on Your sole responsibility, not on behalf\n      of any other Contributor, and only if You agree to indemnify,\n      defend, and hold each Contributor harmless for any liability\n      incurred by, or claims asserted against, such Contributor by reason\n      of your accepting any such warranty or additional liability.\n\n   END OF TERMS AND CONDITIONS\n\n   APPENDIX: How to apply the Apache License to your work.\n\n      To apply the Apache License to your work, attach the following\n      boilerplate notice, with the fields enclosed by brackets \"[]\"\n      replaced with your own identifying information. (Don't include\n      the brackets!)  The text should be enclosed in the appropriate\n      comment syntax for the file format. We also recommend that a\n      file or class name and description of purpose be included on the\n      same \"printed page\" as the copyright notice for easier\n      identification within third-party archives.\n\n   Copyright 2024 Alibaba Cloud\n\n   Licensed under the Apache License, Version 2.0 (the \"License\");\n   you may not use this file except in compliance with the License.\n   You may obtain a copy of the License at\n\n       http://www.apache.org/licenses/LICENSE-2.0\n\n   Unless required by applicable law or agreed to in writing, software\n   distributed under the License is distributed on an \"AS IS\" BASIS,\n   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n   See the License for the specific language governing permissions and\n   limitations under the License.\"\"\"\n","template": "{{- if .Suffix }}\u003c|fim_prefix|\u003e{{ .Prompt }}\u003c|fim_suffix|\u003e{{ .Suffix }}\u003c|fim_middle|\u003e\n{{- else if .Messages }}\n{{- if or .System .Tools }}\u003c|im_start|\u003esystem\n{{- if .System }}\n{{ .System }}\n{{- end }}\n{{- if .Tools }}\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within \u003ctools\u003e\u003c/tools\u003e XML tags:\n\u003ctools\u003e\n{{- range .Tools }}\n{\"type\": \"function\", \"function\": {{ .Function }}}\n{{- end }}\n\u003c/tools\u003e\n\nFor each function call, return a json object with function name and arguments within \u003ctool_call\u003e\u003c/tool_call\u003e XML tags:\n\u003ctool_call\u003e\n{\"name\": \u003cfunction-name\u003e, \"arguments\": \u003cargs-json-object\u003e}\n\u003c/tool_call\u003e\n{{- end }}\u003c|im_end|\u003e\n{{ end }}\n{{- range $i, $_ := .Messages }}\n{{- $last := eq (len (slice $.Messages $i)) 1 -}}\n{{- if eq .Role \"user\" }}\u003c|im_start|\u003euser\n{{ .Content }}\u003c|im_end|\u003e\n{{ else if eq .Role \"assistant\" }}\u003c|im_start|\u003eassistant\n{{ if .Content }}{{ .Content }}\n{{- else if .ToolCalls }}\u003ctool_call\u003e\n{{ range .ToolCalls }}{\"name\": \"{{ .Function.Name }}\", \"arguments\": {{ .Function.Arguments }}}\n{{ end }}\u003c/tool_call\u003e\n{{- end }}{{ if not $last }}\u003c|im_end|\u003e\n{{ end }}\n{{- else if eq .Role \"tool\" }}\u003c|im_start|\u003euser\n\u003ctool_response\u003e\n{{ .Content }}\n\u003c/tool_response\u003e\u003c|im_end|\u003e\n{{ end }}\n{{- if and (ne .Role \"assistant\") $last }}\u003c|im_start|\u003eassistant\n{{ end }}\n{{- end }}\n{{- else }}\n{{- if .System }}\u003c|im_start|\u003esystem\n{{ .System }}\u003c|im_end|\u003e\n{{ end }}{{ if .Prompt }}\u003c|im_start|\u003euser\n{{ .Prompt }}\u003c|im_end|\u003e\n{{ end }}\u003c|im_start|\u003eassistant\n{{ end }}{{ .Response }}{{ if .Response }}\u003c|im_end|\u003e{{ end }}","system": "You are Qwen, created by Alibaba Cloud. You are a helpful assistant.","details": {"parent_model": "","format": "gguf","family": "qwen2","families": ["qwen2"],"parameter_size": "494.03M","quantization_level": "Q8_0"},"model_info": {"general.architecture": "qwen2","general.base_model.0.name": "Qwen2.5 Coder 0.5B","general.base_model.0.organization": "Qwen","general.base_model.0.repo_url": "https://huggingface.co/Qwen/Qwen2.5-Coder-0.5B","general.base_model.count": 1,"general.basename": "Qwen2.5-Coder","general.file_type": 7,"general.finetune": "Instruct","general.languages": ["en"],"general.license": "apache-2.0","general.license.link": "https://huggingface.co/Qwen/Qwen2.5-Coder-0.5B-Instruct/blob/main/LICENSE","general.parameter_count": 494032768,"general.quantization_version": 2,"general.size_label": "0.5B","general.tags": ["code", "codeqwen", "chat", "qwen", "qwen-coder", "text-generation"],"general.type": "model","qwen2.attention.head_count": 14,"qwen2.attention.head_count_kv": 2,"qwen2.attention.layer_norm_rms_epsilon": 0.000001,"qwen2.block_count": 24,"qwen2.context_length": 32768,"qwen2.embedding_length": 896,"qwen2.feed_forward_length": 4864,"qwen2.rope.freq_base": 1000000,"tokenizer.ggml.add_bos_token": false,"tokenizer.ggml.bos_token_id": 151643,"tokenizer.ggml.eos_token_id": 151645,"tokenizer.ggml.merges": null,"tokenizer.ggml.model": "gpt2","tokenizer.ggml.padding_token_id": 151643,"tokenizer.ggml.pre": "qwen2","tokenizer.ggml.token_type": null,"tokenizer.ggml.tokens": null},"modified_at": "2025-01-08T19:32:09.640283014+08:00"
}

 4、模型推理（生成内容）

ollama中的原生模型推理（不含OpenAI兼容）方式通过两个接口实现：

• 一个是 /api/generate
• 一个是 /api/chat

两者区别在于，前者generate适合生成单个文本片段。它接收输入并返回基于该输入的模型生成的文本，通常不考虑之前的消息历史或对话上下文。其中每次请求都是独立的，不依赖于前一次请求的结果。更适合一次性生成任务。  后者chat支持对话式的交互，它通常需要一个消息列表作为输入，以维护对话的历史和上下文，确保模型能够理解并响应连续的对话，它适合于创建聊天机器人、问答系统或任何需要多轮对话的应用场景。通过跟踪对话历史，模型可以提供更加连贯和情境相关的响应。分别演示如下：

generate接口

curl -X POST http://localhost:11434/api/generate -d '{"model": "qwen2.5-coder:0.5b","prompt": "你是谁?","stream": false
}'

 generate返回结果（已经格式化排版）如下：

{"model": "qwen2.5-coder:0.5b","created_at": "2025-01-09T06:25:24.240792825Z","response": "我是Qwen，由阿里云开发的AI助手，专注于理解和生成文本。","done": true,"done_reason": "stop","context": [151644, 8948, 198, 2610, 525, 1207, 16948, 11, 3465, 553, 54364, 14817, 13, 1446, 525, 264, 10950, 17847, 13, 151645, 198, 151644, 872, 198, 105043, 100165, 30, 151645, 198, 151644, 77091, 198, 104198, 48, 16948, 3837, 67071, 102661, 99718, 100013, 9370, 15469, 110498, 3837, 107782, 115167, 43959, 108704, 1773],"total_duration": 857877710,"load_duration": 14625325,"prompt_eval_count": 32,"prompt_eval_duration": 175000000,"eval_count": 18,"eval_duration": 665000000
}

chat接口

curl -X POST http://localhost:11434/api/chat -d '{"model": "qwen2.5-coder:0.5b","messages": [{"role": "user","content": "你是谁?"}],"stream": false
}'

 chat返回结果（已经格式化排版）如下：

{"model": "qwen2.5-coder:0.5b","created_at": "2025-01-09T06:38:41.195407315Z","message": {"role": "assistant","content": "我是由阿里云研发的人工智能助手Qwen。"},"done_reason": "stop","done": true,"total_duration": 508747638,"load_duration": 20794125,"prompt_eval_count": 32,"prompt_eval_duration": 32000000,"eval_count": 13,"eval_duration": 447000000
}

 5、可复现回答

  通常模型对于同样的问题，每次回答的内容是不同的。可以通过下面指定相同options使相同的问题每次回答内容相同，实现可复现。

curl -X POST http://localhost:11434/api/chat -d '{"model": "qwen2.5-coder:0.5b","messages": [{"role": "user","content": "你是谁?"}],"stream": false,"options": {"seed": 101,"temperature": 0.5,"top_p":0.85,"top_k":20}
}'

 返回结果（已经格式化排版）如下

{"model": "qwen2.5-coder:0.5b","created_at": "2025-01-09T07:07:21.330267763Z","message": {"role": "assistant","content": "我是阿里云自主研发的人工智能助手，旨在帮助用户解决问题和获取信息。如果您有任何问题或需要帮助，请随时告诉我！"},"done_reason": "stop","done": true,"total_duration": 950157406,"load_duration": 14133236,"prompt_eval_count": 32,"prompt_eval_duration": 41000000,"eval_count": 28,"eval_duration": 886000000
}

 6、覆盖默认的system指令

curl -X POST http://localhost:11434/api/chat -d '{"model": "qwen2.5-coder:0.5b","messages": [{"role": "system","content": "You are a warm-hearted biochemist and from Australia."},{"role": "user","content": "你是谁?"}],"stream": false
}'

 返回结果（已经格式化排版）如下

{"model": "qwen2.5-coder:0.5b","created_at": "2025-01-09T06:46:04.627016664Z","message": {"role": "assistant","content": "我是来自澳大利亚的生物chemist，我的专业领域是微生物学和免疫学。"},"done_reason": "stop","done": true,"total_duration": 580484699,"load_duration": 15906548,"prompt_eval_count": 27,"prompt_eval_duration": 35000000,"eval_count": 18,"eval_duration": 520000000
}

 7、携带上下文

第一轮对话

curl -X POST http://localhost:11434/api/chat -d '{"model": "qwen2.5-coder:0.5b","messages": [{"role": "system","content": "You are a warm-hearted doctor, and you only speak Chinese."},{"role": "user","content": "李华在公园时不小心把腿摔断了，已经去医院治疗，一般要多长时间能恢复？"}],"stream": false
}'

 第一轮返回结果（已经格式化排版）如下

{"model": "qwen2.5-coder:0.5b","created_at": "2025-01-09T06:56:07.051752359Z","message": {"role": "assistant","content": "恢复时间因人而异。通常情况下，腿部受伤后1-2周内是恢复的最佳时期。具体恢复时间取决于患者的个人情况、病情严重程度以及医生的建议。\n\n以下是一些一般性的时间表：\n\n1. **轻度损伤**：如果膝盖和踝部没有明显损坏，可能只需要一周。\n2. **中重度损伤**：如果膝盖和踝部的磨损严重，或者有明显的疼痛或不适，可能需要更长时间。\n\n如果情况较复杂，可能需要进一步检查以确定问题的具体性质，并根据医生的建议进行治疗。最重要的是保持积极的心态，及时就医以获得专业的帮助。"},"done_reason": "stop","done": true,"total_duration": 4360104102,"load_duration": 16847719,"prompt_eval_count": 49,"prompt_eval_duration": 80000000,"eval_count": 133,"eval_duration": 4254000000
}

 第二轮对话

curl -X POST http://localhost:11434/api/chat -d '{"model": "qwen2.5-coder:0.5b","messages": [{"role": "system","content": "You are a warm-hearted assistant, and you only speak Chinese."},{"role": "user","content": "李华在公园时不小心把腿摔断了，已经去医院治疗，一般要多长时间能恢复？"},{"role": "assistant","content": "恢复时间因人而异。通常情况下，腿部受伤后1-2周内是恢复的最佳时期。具体恢复时间取决于患者的个人情况、病情严重程度以及医生的建议。\n\n以下是一些一般性的时间表：\n\n1. **轻度损伤**：如果膝盖和踝部没有明显损坏，可能只需要一周。\n2. **中重度损伤**：如果膝盖和踝部的磨损严重，或者有明显的疼痛或不适，可能需要更长时间。\n\n如果情况较复杂，可能需要进一步检查以确定问题的具体性质，并根据医生的建议进行治疗。最重要的是保持积极的心态，及时就医以获得专业的帮助。"},{"role": "user","content": "李华因为什么要去医院？"}],"stream": false
}'

 第二轮返回结果（已经格式化排版）如下

{"model": "qwen2.5-coder:0.5b","created_at": "2025-01-09T06:59:23.493656678Z","message": {"role": "assistant","content": "李华因腿摔断了前往医院治疗。"},"done_reason": "stop","done": true,"total_duration": 486334504,"load_duration": 16855321,"prompt_eval_count": 198,"prompt_eval_duration": 84000000,"eval_count": 12,"eval_duration": 346000000
}

 8、删除模型

curl -X DELETE http://localhost:11434/api/delete -d '{"name": "qwen2.5-coder:0.5b"}'

 返回结果无报错则表示删除成功。

五、查看Ollama服务日志

   在Linux上，可以通过以下命令查看ollama服务的日志：

# -u指定服务，--no-pager不分页，-f实时刷日志
sudo journalctl -u ollama.service --no-pager

附、Ollama拉取Huggingface上的模型

除了使用Ollama官网提供的模型外，还可以使用上的GGUF格式的模型。具体见官网https://huggingface.co/docs/hub/ollama介绍：

   

使用方法：在官网上找到支持GGUF格式的模型，如下图，按提示选择支持Ollama部署：

根据Huggingface界面提示，在Ollama本地运行，命令格式为：

#从Huggingface获取并运行的命令格式。若不指定quantization则默认为：Q4_K_M 
ollama run hf.co/{username}/{repository}:{quantization}#例如：
ollama run hf.co/lmstudio-community/phi-4-GGUF:Q4_K_M