环境安装
1.安装conda
curl -O https://repo.anaconda.com/miniconda/Miniconda3-latest-MacOSX-arm64.sh
sh Miniconda3-latest-MacOSX-arm64.sh
2.创建虚拟环境并激活
conda create -n chatglm3 python==3.10.2
conda activate chatglm3
3.安装pytorch-nightly
conda install pytorch torchvision torchaudio -c pytorch-nightly
下载项目文件
下载chatglm3 代码
GitHub地址: https://github.com/THUDM/ChatGLM3
git clone https://github.com/THUDM/ChatGLM3
cd ChatGLM3-main
pip install -r requirement.txt
使用modelscope下载模型文件
下载Chatglm3-6b
huggingface因为众所周知的原因,国内无法访问
使用modelscope下载
https://modelscope.cn/models/ZhipuAI/chatglm3-6b/summary
安装魔塔依赖
pip install modelscope -U
代码中的路径即为要下载的模型,可以自行选择模型下载
model_dir = snapshot_download(‘ZhipuAI/chatglm3-6b’, revision=‘v1.0.1’)
模型默认保存路径为家目录下, ~/.cache/modelscope/hub/ZhipuAI/ChatGLM3-6B/
from modelscope import snapshot_download
model_dir = snapshot_download("ZhipuAI/chatglm3-6b", revision = "v1.0.0")
下载embeddings模型
from modelscope import snapshot_download
model_dir = snapshot_download("BAAI/bge-m3", revision = "v1.0.0")
使用模型
修改web_demo2.py中的device为"mps"
DEVICE = "mps"
MODEL_PATH = os.environ.get('MODEL_PATH', 'local_model_path')
model = AutoModel.from_pretrained(MODEL_PATH, trust_remote_code=True, device_map="auto").eval()
# web_demo2.py 运行
streamlit run basic_demo/web_demo2.py
演示页面
浏览器输入http://127.0.0.1:7860/
使用openai格式的api
进入项目的openai_api_demo目录
cd openai_api_demo
修改api_server.py文件
# model = AutoModel.from_pretrained(MODEL_PATH, trust_remote_code=True, device_map="auto").eval()
model = AutoModel.from_pretrained(MODEL_PATH, trust_remote_code=True).to('mps')
# load Embedding
embedding_model = SentenceTransformer(EMBEDDING_PATH, device="mps")
启动fastapi服务
python3 api_server.py
使用openai格式的api
import openai
openai.api_base = "http://192.168.0.xx:80/v1"
# Enter any non-empty API key to pass the client library's check.
openai.api_key = "xxx"
# Enter any non-empty model name to pass the client library's check.
completion = openai.ChatCompletion.create(
model="chatglm-6b",
messages=[
{"role": "user", "content": "你好"},
],
stream=False,
)
print(completion.choices[0].message.content)
# 你好👋!我是人工智能助手 ChatGLM-6B,很高兴见到你,欢迎问我任何问题。