docker 本地部署大模型(ollama)

docker 安装 ollama

docker search ollama
docker pull ollama/ollama

###docker下载ollama部署
docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama

### 下载模型
docker exec -it ollama ollama pull llama3

### 交互式运行模型
 docker exec -it ollama ollama run llama3

注意有坑:docker -v ollama:/root/.ollama
此处由于挂载目录使用了相对路径,所以本地文件夹位于 /var/lib/docker/volumes/ollama
而非运行命令的相对路径

测试 api

$ curl http://localhost:14444/api/generate -d '{ "model": "llama3", "prompt": "請列出五樣台灣美食", "stream": true, "options": { "seed": 123, "top_k": 20, "top_p": 0.9, "temperature": 0 } }'
{"model":"llama3","created_at":"2024-07-08T03:49:16.611142384Z","response":"Here","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:16.758150085Z","response":" are","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:16.903960154Z","response":" ","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:17.048831268Z","response":"5","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:17.192653835Z","response":" must","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:17.340323632Z","response":"-","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:17.490090786Z","response":"try","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:17.630377981Z","response":" Taiwanese","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:17.79671948Z","response":" dishes","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:17.947959672Z","response":":\n\n","done":false}
.......
{"model":"llama3","created_at":"2024-07-08T03:49:51.670524112Z","response":" dishes","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:51.80466663Z","response":" showcase","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:51.953821627Z","response":" the","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:52.112141942Z","response":" diversity","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:52.274051058Z","response":" and","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:52.41761068Z","response":" richness","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:52.571736412Z","response":" of","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:52.719734804Z","response":" Taiwanese","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:52.857012216Z","response":" cuisine","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:52.998817903Z","response":",","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:53.166612385Z","response":" which","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:53.342996684Z","response":" often","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:53.510572997Z","response":" combines","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:53.668246714Z","response":" Chinese","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:53.836925764Z","response":",","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:53.986648055Z","response":" Japanese","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:54.13517455Z","response":",","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:54.283114108Z","response":" and","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:54.443445171Z","response":" indigenous","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:54.603118958Z","response":" influences","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:54.76832015Z","response":".","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:54.931017326Z","response":" Enjoy","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:55.083288072Z","response":"!","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:55.533841685Z","response":" 🍴","done":false}
{"model":"llama3","created_at":"2024-07-08T03:49:55.671222985Z","response":"","done":true,"done_reason":"stop","context":[128006,882,128007,271,109311,32218,20834,76208,112595,117342,58666,102456,128009,128006,78191,128007,271,8586,527,220,20,2011,12,1568,94037,26863,1473,16,13,3146,3513,830,452,31707,61434,1754,96618,362,11670,94037,12269,11,25309,46895,273,19724,374,264,82651,323,94631,45993,1903,449,6435,23283,11059,25309,11,61260,11,323,24822,627,17,13,3146,46,22604,507,2727,1169,1754,96618,362,5526,8761,3691,304,29389,11,297,22604,297,2727,10145,527,73624,41951,19335,10409,449,628,1538,297,100163,11,70181,11,323,6307,45697,13,328,2841,449,264,10437,323,94760,19737,627,18,13,3146,626,39134,350,1073,84,1754,96618,49386,369,1202,281,2234,306,22843,11,357,39134,80247,374,264,96080,29839,17937,2027,430,596,5655,2269,4588,311,39143,13,1102,596,3629,10434,449,264,86921,19737,323,649,387,5115,57407,4999,19,13,3146,33,93613,85034,97685,1754,96618,362,69130,323,9257,12269,11,20716,4147,36167,36517,374,6435,23283,11059,304,264,21655,315,29839,19737,11,13465,11,323,49659,3156,279,13339,374,28682,323,17503,10980,6847,627,20,13,3146,38,4381,426,3524,1754,96618,7429,3967,439,94037,63452,11,1709,64,103253,527,4179,3690,293,11099,10409,449,20716,4147,36167,11,3820,839,24822,11,323,42393,13,362,18406,323,37154,40459,477,15496,382,9673,26863,35883,279,20057,323,90030,315,94037,36105,11,902,3629,33511,8620,11,11002,11,323,39678,34453,13,23485,0,11410,235,112,128009],"total_duration":43461066356,"load_duration":3482212558,"prompt_eval_count":17,"prompt_eval_duration":916597000,"eval_count":260,"eval_duration":39060126000}

python + ollama

from ollama import Client 
client = Client(host=f'http://{IP_ADDRESS}:14444') 
response = client.chat(model='llama3', messages=[ { 'role': 'user', 'content': 'Why is the sky blue?', }, ])
response 
{'model': 'llama3',
 'created_at': '2024-07-08T04:05:41.013483539Z',
 'message': {'role': 'assistant',
  'content': "The sky appears blue because of a phenomenon called Rayleigh scattering, which is named after the British physicist Lord Rayleigh. Here's what happens:\n\n1. **Sunlight**: When sunlight enters Earth's atmosphere, it contains all the colors of the visible spectrum (red, orange, yellow, green, blue, indigo, and violet).\n2. **Molecules**: The atmosphere is made up of tiny molecules of gases like nitrogen (N2) and oxygen (O2). These molecules are much smaller than the wavelength of light.\n3. **Scattering**: When sunlight hits these small molecules, it scatters in all directions. This scattering effect is more pronounced for shorter wavelengths, like blue and violet light, which have a higher frequency than longer wavelengths, like red and orange light.\n4. **Blue dominance**: As a result of this scattering, the blue light with its shorter wavelength is dispersed throughout the atmosphere, reaching our eyes from all parts of the sky. This is why the sky appears blue during the daytime, especially when the sun is overhead.\n\nThe same principle applies to other colors, but with different intensities:\n\n* **Red and orange**: These longer wavelengths are less affected by scattering and reach our eyes more directly, which is why we see them as dominant at sunrise and sunset.\n* **Green and yellow**: These wavelengths are scattered similarly to blue light, but their intensity decreases with distance from the sun, making them less visible in the sky.\n\nSo, to summarize: the sky appears blue because of the scattering of sunlight by tiny molecules in the atmosphere, which favors shorter wavelengths like blue and violet."},
 'done_reason': 'stop',
 'done': True,
 'total_duration': 52984861518,
 'load_duration': 3257720984,
 'prompt_eval_count': 15,
 'prompt_eval_duration': 829317000,
 'eval_count': 325,
 'eval_duration': 4888172200

流式输出

import ollama

stream = client.chat(
    model='llama3',
    messages=[{'role': 'user', 'content': 'Why is the sky blue?'}],
    stream=True,
)

for chunk in stream:
    print(chunk['message']['content'], end='', flush=True)

在这里插入图片描述

参考

  • https://github.com/ollama/ollama-python/tree/main/

相关推荐

  1. Ollama+AnythingLLM搭建部署本地模型AI知识库

    2024-07-09 17:28:03       58 阅读
  2. ollama集成open-webui本地部署模型应用

    2024-07-09 17:28:03       15 阅读

最近更新

  1. 什么是 MLPerf?

    2024-07-09 17:28:03       0 阅读
  2. Docker

    2024-07-09 17:28:03       0 阅读
  3. 代码改进,模型优化,强化深度学习

    2024-07-09 17:28:03       0 阅读
  4. python 基础知识点(一)

    2024-07-09 17:28:03       0 阅读
  5. Python利用psutil库进行监控进程和资源

    2024-07-09 17:28:03       0 阅读
  6. SpringBoot RestHighLevelClient 按版本更新

    2024-07-09 17:28:03       0 阅读
  7. 跨域问题几种解决方法

    2024-07-09 17:28:03       0 阅读
  8. Python面试整理-文件处理

    2024-07-09 17:28:03       0 阅读

热门阅读

  1. Ubuntu下Qt-5.12.9创建快捷方式到桌面

    2024-07-09 17:28:03       9 阅读
  2. ArkTs基础入门

    2024-07-09 17:28:03       10 阅读
  3. 代码随想录Day74(图论Part10)

    2024-07-09 17:28:03       10 阅读
  4. 如何使一个盒子水平垂直居中(常用的)

    2024-07-09 17:28:03       5 阅读
  5. mybatis用注解替换xml,不再写.xml了

    2024-07-09 17:28:03       8 阅读
  6. Docker

    Docker

    2024-07-09 17:28:03      7 阅读
  7. 服务器安装多个Tomcat

    2024-07-09 17:28:03       7 阅读
  8. 玩转springboot之springboot定制化tomcat

    2024-07-09 17:28:03       9 阅读