大模型学习应用 1：用 itrex 创新高效实现 LLM 的部署和微调

用 itrex 创新高效实现 LLM 的部署和微调 - 项目作业

准备工作

注意本项目是在和鲸的虚拟环境上运行的，和本地实际部署流程可能会有差异 ，项目的难点在于调整设置prompt,不过文档中配置安装会占据较大篇幅，直接浏览代码块就好。

计算资源：腾讯云南京CPU16核64G
使用镜像：3.11.8数据科学镜像

项目原链接
参考项目

Task 1 完成在线环境的工具包安装，包含 基础环境包、Extension for Transformers 包、加速计算包

# Your Answer：环境安装代码
# 安装Itrex在CPU环境下使用所需的系统环境
!sudo apt-get update
!sudo apt-get install -y ffmpeg
!sudo apt-get install -y libgl1-mesa-glx libgl1-mesa-dev
!sudo apt-get install -y libsm6 libxext6

Get:1 http://archive.ubuntu.com/ubuntu jammy InRelease [270 kB]
Get:2 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB]
Get:3 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [44.7 kB]
Get:4 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB]
Get:5 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2717 kB]
Get:6 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB]
Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 Packages [1792 kB]
Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 Packages [17.5 MB]
Get:9 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [2071 kB]
Get:10 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [1128 kB]
Get:11 http://archive.ubuntu.com/ubuntu jammy/multiverse amd64 Packages [266 kB]
Get:12 http://archive.ubuntu.com/ubuntu jammy/restricted amd64 Packages [164 kB]
Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [51.8 kB]
Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2338 kB]
Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2795 kB]
Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1418 kB]
Get:17 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [81.0 kB]
Get:18 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [33.7 kB]
Fetched 33.0 MB in 10s (3172 kB/s)                                             
Reading package lists... Done
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
ffmpeg is already the newest version (7:4.4.2-0ubuntu0.22.04.1).
0 upgraded, 0 newly installed, 0 to remove and 41 not upgraded.
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following additional packages will be installed:libegl-dev libegl-mesa0 libegl1 libgl-dev libgles-dev libgles1 libgles2libglvnd-core-dev libglvnd-dev libglx-dev libopengl-dev libopengl0libpthread-stubs0-dev libx11-dev libxau-dev libxcb1-dev libxdmcp-devx11proto-dev xorg-sgml-doctools xtrans-dev
Suggested packages:libx11-doc libxcb-doc
The following NEW packages will be installed:libegl-dev libegl-mesa0 libegl1 libgl-dev libgl1-mesa-dev libgl1-mesa-glxlibgles-dev libgles1 libgles2 libglvnd-core-dev libglvnd-dev libglx-devlibopengl-dev libopengl0 libpthread-stubs0-dev libx11-dev libxau-devlibxcb1-dev libxdmcp-dev x11proto-dev xorg-sgml-doctools xtrans-dev
0 upgraded, 22 newly installed, 0 to remove and 41 not upgraded.
Need to get 1985 kB of archives.
After this operation, 8967 kB of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libegl-mesa0 amd64 23.2.1-1ubuntu3.1~22.04.2 [118 kB]
Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 libegl1 amd64 1.4.0-1 [28.6 kB]
Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 xorg-sgml-doctools all 1:1.11-1.1 [10.9 kB]
Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 x11proto-dev all 2021.5-1 [604 kB]
Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 libxau-dev amd64 1:1.0.9-1build5 [9724 B]
Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 libxdmcp-dev amd64 1:1.1.3-0ubuntu5 [26.5 kB]
Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 xtrans-dev all 1.4.0-1 [68.9 kB]
Get:8 http://archive.ubuntu.com/ubuntu jammy/main amd64 libpthread-stubs0-dev amd64 0.4-1build2 [5516 B]
Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 libxcb1-dev amd64 1.14-3ubuntu3 [86.5 kB]
Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libx11-dev amd64 2:1.7.5-1ubuntu0.3 [744 kB]
Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 libglx-dev amd64 1.4.0-1 [14.1 kB]
Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libgl-dev amd64 1.4.0-1 [101 kB]
Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libegl-dev amd64 1.4.0-1 [18.0 kB]
Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libgl1-mesa-glx amd64 23.0.4-0ubuntu1~22.04.1 [5584 B]
Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libgles1 amd64 1.4.0-1 [11.5 kB]
Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 libgles2 amd64 1.4.0-1 [18.0 kB]
Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 libgles-dev amd64 1.4.0-1 [49.4 kB]
Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 libopengl0 amd64 1.4.0-1 [36.5 kB]
Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 libopengl-dev amd64 1.4.0-1 [3400 B]
Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 libglvnd-core-dev amd64 1.4.0-1 [12.7 kB]
Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 libglvnd-dev amd64 1.4.0-1 [3162 B]
Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libgl1-mesa-dev amd64 23.2.1-1ubuntu3.1~22.04.2 [6842 B]
Fetched 1985 kB in 4s (460 kB/s)           
debconf: delaying package configuration, since apt-utils is not installed
Selecting previously unselected package libegl-mesa0:amd64.
(Reading database ... 49714 files and directories currently installed.)
Preparing to unpack .../00-libegl-mesa0_23.2.1-1ubuntu3.1~22.04.2_amd64.deb ...
Unpacking libegl-mesa0:amd64 (23.2.1-1ubuntu3.1~22.04.2) ...
Selecting previously unselected package libegl1:amd64.
Preparing to unpack .../01-libegl1_1.4.0-1_amd64.deb ...
Unpacking libegl1:amd64 (1.4.0-1) ...
Selecting previously unselected package xorg-sgml-doctools.
Preparing to unpack .../02-xorg-sgml-doctools_1%3a1.11-1.1_all.deb ...
Unpacking xorg-sgml-doctools (1:1.11-1.1) ...
Selecting previously unselected package x11proto-dev.
Preparing to unpack .../03-x11proto-dev_2021.5-1_all.deb ...
Unpacking x11proto-dev (2021.5-1) ...
Selecting previously unselected package libxau-dev:amd64.
Preparing to unpack .../04-libxau-dev_1%3a1.0.9-1build5_amd64.deb ...
Unpacking libxau-dev:amd64 (1:1.0.9-1build5) ...
Selecting previously unselected package libxdmcp-dev:amd64.
Preparing to unpack .../05-libxdmcp-dev_1%3a1.1.3-0ubuntu5_amd64.deb ...
Unpacking libxdmcp-dev:amd64 (1:1.1.3-0ubuntu5) ...
Selecting previously unselected package xtrans-dev.
Preparing to unpack .../06-xtrans-dev_1.4.0-1_all.deb ...
Unpacking xtrans-dev (1.4.0-1) ...
Selecting previously unselected package libpthread-stubs0-dev:amd64.
Preparing to unpack .../07-libpthread-stubs0-dev_0.4-1build2_amd64.deb ...
Unpacking libpthread-stubs0-dev:amd64 (0.4-1build2) ...
Selecting previously unselected package libxcb1-dev:amd64.
Preparing to unpack .../08-libxcb1-dev_1.14-3ubuntu3_amd64.deb ...
Unpacking libxcb1-dev:amd64 (1.14-3ubuntu3) ...
Selecting previously unselected package libx11-dev:amd64.
Preparing to unpack .../09-libx11-dev_2%3a1.7.5-1ubuntu0.3_amd64.deb ...
Unpacking libx11-dev:amd64 (2:1.7.5-1ubuntu0.3) ...
Selecting previously unselected package libglx-dev:amd64.
Preparing to unpack .../10-libglx-dev_1.4.0-1_amd64.deb ...
Unpacking libglx-dev:amd64 (1.4.0-1) ...
Selecting previously unselected package libgl-dev:amd64.
Preparing to unpack .../11-libgl-dev_1.4.0-1_amd64.deb ...
Unpacking libgl-dev:amd64 (1.4.0-1) ...
Selecting previously unselected package libegl-dev:amd64.
Preparing to unpack .../12-libegl-dev_1.4.0-1_amd64.deb ...
Unpacking libegl-dev:amd64 (1.4.0-1) ...
Selecting previously unselected package libgl1-mesa-glx:amd64.
Preparing to unpack .../13-libgl1-mesa-glx_23.0.4-0ubuntu1~22.04.1_amd64.deb ...
Unpacking libgl1-mesa-glx:amd64 (23.0.4-0ubuntu1~22.04.1) ...
Selecting previously unselected package libgles1:amd64.
Preparing to unpack .../14-libgles1_1.4.0-1_amd64.deb ...
Unpacking libgles1:amd64 (1.4.0-1) ...
Selecting previously unselected package libgles2:amd64.
Preparing to unpack .../15-libgles2_1.4.0-1_amd64.deb ...
Unpacking libgles2:amd64 (1.4.0-1) ...
Selecting previously unselected package libgles-dev:amd64.
Preparing to unpack .../16-libgles-dev_1.4.0-1_amd64.deb ...
Unpacking libgles-dev:amd64 (1.4.0-1) ...
Selecting previously unselected package libopengl0:amd64.
Preparing to unpack .../17-libopengl0_1.4.0-1_amd64.deb ...
Unpacking libopengl0:amd64 (1.4.0-1) ...
Selecting previously unselected package libopengl-dev:amd64.
Preparing to unpack .../18-libopengl-dev_1.4.0-1_amd64.deb ...
Unpacking libopengl-dev:amd64 (1.4.0-1) ...
Selecting previously unselected package libglvnd-core-dev:amd64.
Preparing to unpack .../19-libglvnd-core-dev_1.4.0-1_amd64.deb ...
Unpacking libglvnd-core-dev:amd64 (1.4.0-1) ...
Selecting previously unselected package libglvnd-dev:amd64.
Preparing to unpack .../20-libglvnd-dev_1.4.0-1_amd64.deb ...
Unpacking libglvnd-dev:amd64 (1.4.0-1) ...
Selecting previously unselected package libgl1-mesa-dev:amd64.
Preparing to unpack .../21-libgl1-mesa-dev_23.2.1-1ubuntu3.1~22.04.2_amd64.deb ...
Unpacking libgl1-mesa-dev:amd64 (23.2.1-1ubuntu3.1~22.04.2) ...
Setting up libglvnd-core-dev:amd64 (1.4.0-1) ...
Setting up libpthread-stubs0-dev:amd64 (0.4-1build2) ...
Setting up libopengl0:amd64 (1.4.0-1) ...
Setting up xtrans-dev (1.4.0-1) ...
Setting up libegl-mesa0:amd64 (23.2.1-1ubuntu3.1~22.04.2) ...
Setting up libgles2:amd64 (1.4.0-1) ...
Setting up libgles1:amd64 (1.4.0-1) ...
Setting up libgl1-mesa-glx:amd64 (23.0.4-0ubuntu1~22.04.1) ...
Setting up libegl1:amd64 (1.4.0-1) ...
Setting up xorg-sgml-doctools (1:1.11-1.1) ...
Setting up libopengl-dev:amd64 (1.4.0-1) ...
Setting up x11proto-dev (2021.5-1) ...
Setting up libxau-dev:amd64 (1:1.0.9-1build5) ...
Setting up libxdmcp-dev:amd64 (1:1.1.3-0ubuntu5) ...
Setting up libxcb1-dev:amd64 (1.14-3ubuntu3) ...
Setting up libx11-dev:amd64 (2:1.7.5-1ubuntu0.3) ...
Setting up libglx-dev:amd64 (1.4.0-1) ...
Setting up libgl-dev:amd64 (1.4.0-1) ...
Setting up libegl-dev:amd64 (1.4.0-1) ...
Setting up libgles-dev:amd64 (1.4.0-1) ...
Setting up libglvnd-dev:amd64 (1.4.0-1) ...
Setting up libgl1-mesa-dev:amd64 (23.2.1-1ubuntu3.1~22.04.2) ...
Processing triggers for libc-bin (2.35-0ubuntu3.6) ...
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
libsm6 is already the newest version (2:1.2.3-1build2).
libsm6 set to manually installed.
libxext6 is already the newest version (2:1.3.4-1build1).
libxext6 set to manually installed.
0 upgraded, 0 newly installed, 0 to remove and 41 not upgraded.

# 安装所需的第三方库，包含环境依赖包和可选用于加速计算的依赖包
!conda install -y pytorch==2.3.0 torchvision==0.18.0 torchaudio==2.3.0 cpuonly -c pytorch
!pip install cmake -i https://mirrors.cloud.tencent.com/pypi/simple
!pip install ninja -i https://mirrors.cloud.tencent.com/pypi/simple
!pip install neural_speed -i https://mirrors.cloud.tencent.com/pypi/simple
!pip install intel-extension-for-transformers -i https://mirrors.cloud.tencent.com/pypi/simple
!pip install modelscope -i https://mirrors.cloud.tencent.com/pypi/simple
!pip install transformers -i https://mirrors.cloud.tencent.com/pypi/simple
!pip install pyOpenSSL --upgrade -i https://mirrors.cloud.tencent.com/pypi/simple
!pip install sentencepiece --upgrade -i https://mirrors.cloud.tencent.com/pypi/simple
!pip install xformers -i https://mirrors.cloud.tencent.com/pypi/simple
!pip install accelerate -i https://mirrors.cloud.tencent.com/pypi/simple
!pip install tiktoken -i https://mirrors.cloud.tencent.com/pypi/simple
!pip install transformers_stream_generator -i https://mirrors.cloud.tencent.com/pypi/simple

    Channels:- pytorch- conda-forgePlatform: linux-64Collecting package metadata (repodata.json): doneSolving environment: done==> WARNING: A newer version of conda exists. <==current version: 24.3.0latest version: 24.5.0Please update conda by running$ conda update -n base -c conda-forge conda​    ## Package Plan ##environment location: /opt/condaadded / updated specs:- cpuonly- pytorch==2.3.0- torchaudio==2.3.0- torchvision==0.18.0The following packages will be downloaded:package                    |            build---------------------------|-----------------ca-certificates-2024.7.4   |       hbcca054_0         151 KB  conda-forgecertifi-2024.7.4           |     pyhd8ed1ab_0         156 KB  conda-forgecpuonly-2.0                |                0           2 KB  pytorchffmpeg-4.3                 |       hf484d3e_0         9.9 MB  pytorchfilelock-3.15.4            |     pyhd8ed1ab_0          17 KB  conda-forgegnutls-3.6.13              |       h85f3911_1         2.0 MB  conda-forgelame-3.100                 |    h166bdaf_1003         496 KB  conda-forgelibabseil-20240116.2       | cxx17_he02047a_1         1.2 MB  conda-forgelibtorch-2.3.0             |cpu_generic_h970db74_1        47.7 MB  conda-forgenettle-3.6                 |       he412f7d_0         6.5 MB  conda-forgeopenh264-2.1.1             |       h780b84a_0         1.5 MB  conda-forgeopenssl-3.3.1              |       h4bc722e_2         2.8 MB  conda-forgepytorch-2.3.0              |cpu_generic_py311h8ca351a_1        31.5 MB  conda-forgepytorch-mutex-1.0          |              cpu           3 KB  pytorchsleef-3.6.1                |       h3400bea_1         975 KB  conda-forgetorchaudio-2.3.0           |        py311_cpu         5.1 MB  pytorchtorchvision-0.18.0         |        py311_cpu         7.0 MB  pytorch------------------------------------------------------------Total:       116.9 MBThe following NEW packages will be INSTALLED:cpuonly            pytorch/noarch::cpuonly-2.0-0 ffmpeg             pytorch/linux-64::ffmpeg-4.3-hf484d3e_0 filelock           conda-forge/noarch::filelock-3.15.4-pyhd8ed1ab_0 gnutls             conda-forge/linux-64::gnutls-3.6.13-h85f3911_1 lame               conda-forge/linux-64::lame-3.100-h166bdaf_1003 libtorch           conda-forge/linux-64::libtorch-2.3.0-cpu_generic_h970db74_1 nettle             conda-forge/linux-64::nettle-3.6-he412f7d_0 openh264           conda-forge/linux-64::openh264-2.1.1-h780b84a_0 pytorch            conda-forge/linux-64::pytorch-2.3.0-cpu_generic_py311h8ca351a_1 pytorch-mutex      pytorch/noarch::pytorch-mutex-1.0-cpu sleef              conda-forge/linux-64::sleef-3.6.1-h3400bea_1 torchaudio         pytorch/linux-64::torchaudio-2.3.0-py311_cpu torchvision        pytorch/linux-64::torchvision-0.18.0-py311_cpu The following packages will be UPDATED:ca-certificates                       2024.2.2-hbcca054_0 --> 2024.7.4-hbcca054_0 certifi                             2024.2.2-pyhd8ed1ab_0 --> 2024.7.4-pyhd8ed1ab_0 libabseil                     20240116.1-cxx17_h59595ed_2 --> 20240116.2-cxx17_he02047a_1 openssl                                  3.2.1-hd590300_1 --> 3.3.1-h4bc722e_2 ​    Downloading and Extracting Packages:libtorch-2.3.0       | 47.7 MB   |                                       |   0% pytorch-2.3.0        | 31.5 MB   |                                       |   0% [Affmpeg-4.3           | 9.9 MB    |                                       |   0% [A[Atorchvision-0.18.0   | 7.0 MB    |                                       |   0% [A[A[A​    
​    nettle-3.6           | 6.5 MB    |                                       |   0% [A[A[A[A​    
​    
​    torchaudio-2.3.0     | 5.1 MB    |                                       |   0% [A[A[A[A[A​    
​    
​    
​    openssl-3.3.1        | 2.8 MB    |                                       |   0% [A[A[A[A[A[A​    
​    
​    
​    
​    gnutls-3.6.13        | 2.0 MB    |                                       |   0% [A[A[A[A[A[A[A​    
​    
​    
​    
​    
​    openh264-2.1.1       | 1.5 MB    |                                       |   0% [A[A[A[A[A[A[A[A​    
​    
​    
​    
​    
​    
​    libabseil-20240116.2 | 1.2 MB    |                                       |   0% [A[A[A[A[A[A[A[A[A​    
​    
​    
​    
​    
​    
​    
​    sleef-3.6.1          | 975 KB    |                                       |   0% [A[A[A[A[A[A[A[A[A[A​    
​    
​    
​    
​    
​    
​    
​    
​    lame-3.100           | 496 KB    |                                       |   0% [A[A[A[A[A[A[A[A[A[A[A​    
​    
​    
​    
​    
​    
​    
​    
​    
​    certifi-2024.7.4     | 156 KB    |                                       |   0% [A[A[A[A[A[A[A[A[A[A[A[A​    
​    
​    
​    
​    
​    
​    
​    
​    
​    
​    ca-certificates-2024 | 151 KB    |                                       |   0% [A[A[A[A[A[A[A[A[A[A[A[A[A​    
​    
​    
​    
​    
​    
​    
​    
​    
​    
​    
​    filelock-3.15.4      | 17 KB     |                                       |   0% [A[A[A[A[A[A[A[A[A[A[A[A[A[A​    
​    
​    
​    
​    
​    
​    
​    
​    
​    
​    
​    
​    pytorch-mutex-1.0    | 3 KB      |                                       |   0% [A[A[A[A[A[A[A[A[A[A[A[A[A[A[A​    
​    
​    
​    
​    
​    
​    
​    
​    
​    
​    
​    
​    
​    cpuonly-2.0          | 2 KB      |                                       |   0% [A[A[A[A[A[A[A[A[A[A[A[A[A[A[A[A​    torchvision-0.18.0   | 7.0 MB    |                                       |   0% [A[A[Affmpeg-4.3           | 9.9 MB    |                                       |   0% [A[A​    
​    libtorch-2.3.0       | 47.7 MB   |                                       |   0% [A[A[A[Apytorch-2.3.0        | 31.5 MB   |                                       |   0% [Atorchvision-0.18.0   | 7.0 MB    | 2                                     |   1% [A[A[Affmpeg-4.3           | 9.9 MB    | 1                                     |   0% [A[A​    
​    libtorch-2.3.0       | 47.7 MB   |                                       |   0% [A[A[A[Apytorch-2.3.0        | 31.5 MB   |                                       |   0% [Atorchvision-0.18.0   | 7.0 MB    | 5                                     |   2% [A[A[A​    
​    nettle-3.6           | 6.5 MB    | 6                                     |   2% [A[A[A[Alibtorch-2.3.0       | 47.7 MB   |                                       |   0% [A[A​    torchvision-0.18.0   | 7.0 MB    | #1                                    |   3% [A[A[Apytorch-2.3.0        | 31.5 MB   | 1                                     |   0% [Affmpeg-4.3           | 9.9 MB    | 7                                     |   2% [A[A​    libtorch-2.3.0       | 47.7 MB   | 1                                     |   0% [A[A[A[A​    torchvision-0.18.0   | 7.0 MB    | #8                                    |   5% [A[A[Apytorch-2.3.0        | 31.5 MB   | 2                                     |   1% [Affmpeg-4.3           | 9.9 MB    | #2                                    |   3% [A[A​    libtorch-2.3.0       | 47.7 MB   | 2                                     |   1% [A[A[A[A​    torchvision-0.18.0   | 7.0 MB    | ###1                                  |   8% [A[A[Affmpeg-4.3           | 9.9 MB    | ##                                    |   5% [A[A​    
​    libtorch-2.3.0       | 47.7 MB   | 3                                     |   1% [A[A[A[A​    torchvision-0.18.0   | 7.0 MB    | #####2                                |  14% [A[A[Affmpeg-4.3           | 9.9 MB    | ###4                                  |   9% [A[A​    
​    nettle-3.6           | 6.5 MB    | #####7                                |  15% [A[A[A[Alibtorch-2.3.0       | 47.7 MB   | 6                                     |   2% [Atorchvision-0.18.0   | 7.0 MB    | ########6                             |  23% [A[A[Affmpeg-4.3           | 9.9 MB    | #####5                                |  15% [A[A​    
​    nettle-3.6           | 6.5 MB    | #########4                            |  26% [A[A[A[A​    torchvision-0.18.0   | 7.0 MB    | ##############5                       |  39% [A[A[Apytorch-2.3.0        | 31.5 MB   | 3                                     |   1% [Affmpeg-4.3           | 9.9 MB    | #########2                            |  25% [A[A​    nettle-3.6           | 6.5 MB    | ###############7                      |  43% [A[A[A[A​    torchvision-0.18.0   | 7.0 MB    | #######################9              |  65% [A[A[Affmpeg-4.3           | 9.9 MB    | ###############1                      |  41% [A[A​    
​    nettle-3.6           | 6.5 MB    | #########################9            |  70% [A[A[A[Alibtorch-2.3.0       | 47.7 MB   | 9                                     |   3% [Affmpeg-4.3           | 9.9 MB    | ########################5             |  66% [A[Alibtorch-2.3.0       | 47.7 MB   | ##8                                   |   8% [Apytorch-2.3.0        | 31.5 MB   | ##3                                   |   6% [A​    
​    torchaudio-2.3.0     | 5.1 MB    | 1                                     |   0% [A[A[A[A[A​    
​    
​    
​    libtorch-2.3.0       | 47.7 MB   | #####7                                |  16% [A[A[A[A[A[A​    
​    
​    
​    
​    libtorch-2.3.0       | 47.7 MB   | ########3                             |  23% [A[A[A[A[A[A[Apytorch-2.3.0        | 31.5 MB   | ##9                                   |   8% [A​    
​    
​    
​    
​    openh264-2.1.1       | 1.5 MB    | 3                                     |   1% [A[A[A[A[A[A[A[A​    
​    
​    
​    
​    
​    
​    libtorch-2.3.0       | 47.7 MB   | ############3                         |  34% [A[A[A[A[A[A[A[A[A​    
​    
​    
​    
​    gnutls-3.6.13        | 2.0 MB    | ###################3                  |  52% [A[A[A[A[A[A[Alibtorch-2.3.0       | 47.7 MB   | ################1                     |  44% [A​    
​    
​    
​    
​    
​    
​    sleef-3.6.1          | 975 KB    | 6                                     |   2% [A[A[A[A[A[A[A[A[A[A​    
​    
​    
​    
​    
​    
​    
​    
​    lame-3.100           | 496 KB    | #1                                    |   3% [A[A[A[A[A[A[A[A[A[A[A​    
​    
​    
​    
​    
​    
​    
​    
​    
​    certifi-2024.7.4     | 156 KB    | ###8                                  |  10% [A[A[A[A[A[A[A[A[A[A[A[Apytorch-2.3.0        | 31.5 MB   | #####3                                |  14% [A​    
​    
​    
​    
​    
​    
​    
​    
​    
​    ca-certificates-2024 | 151 KB    | ###9                                  |  11% [A[A[A[A[A[A[A[A[A[A[A[A[A​    
​    
​    
​    
​    
​    
​    
​    
​    
​    
​    
​    filelock-3.15.4      | 17 KB     | ##################################4   |  93% [A[A[A[A[A[A[A[A[A[A[A[A[A[Apytorch-2.3.0        | 31.5 MB   | ######5                               |  18% [A​    
​    
​    
​    
​    
​    
​    
​    
​    
​    
​    
​    libtorch-2.3.0       | 47.7 MB   | ######################8               |  62% [A[A[A[A[A[A[A[A[A[A[A[A[A[A[A​    
​    
​    
​    
​    
​    
​    
​    
​    
​    
​    
​    
​    
​    cpuonly-2.0          | 2 KB      | ##################################### | 100% [A[A[A[A[A[A[A[A[A[A[A[A[A[A[A[Apytorch-2.3.0        | 31.5 MB   | ############6                         |  34% [Atorchvision-0.18.0   | 7.0 MB    | ##################################### | 100% [A[A[A​    torchvision-0.18.0   | 7.0 MB    | ##################################### | 100% [A[A[A​    
​    nettle-3.6           | 6.5 MB    | ##################################### | 100% [A[A[A[A​    
​    libtorch-2.3.0       | 47.7 MB   | #########################5            |  69% [A[A[A[A​    
​    
​    
​    openssl-3.3.1        | 2.8 MB    | ##################################### | 100% [A[A[A[A[A[A​    
​    
​    
​    libtorch-2.3.0       | 47.7 MB   | ###################################1  |  95% [A[A[A[A[A[A​    
​    
​    
​    
​    
​    openh264-2.1.1       | 1.5 MB    | ##################################### | 100% [A[A[A[A[A[A[A[A​    
​    
​    
​    
​    
​    openh264-2.1.1       | 1.5 MB    | ##################################### | 100% [A[A[A[A[A[A[A[Apytorch-2.3.0        | 31.5 MB   | ##############4                       |  39% [Apytorch-2.3.0        | 31.5 MB   | ######################6               |  61% [Affmpeg-4.3           | 9.9 MB    | ##################################### | 100% [A[Affmpeg-4.3           | 9.9 MB    | ##################################### | 100% [A[Apytorch-2.3.0        | 31.5 MB   | ##########################            |  70% [A​    
​    
​    
​    
​    
​    libabseil-20240116.2 | 1.2 MB    | ##################################### | 100% [A[A[A[A[A[A[A[A[A​    
​    
​    
​    
​    
​    
​    libabseil-20240116.2 | 1.2 MB    | ##################################### | 100% [A[A[A[A[A[A[A[A[A​    
​    
​    
​    
​    
​    
​    
​    sleef-3.6.1          | 975 KB    | ##################################### | 100% [A[A[A[A[A[A[A[A[A[A​    
​    
​    
​    
​    
​    
​    
​    sleef-3.6.1          | 975 KB    | ##################################### | 100% [A[A[A[A[A[A[A[A[A[A​    
​    
​    
​    
​    
​    
​    
​    
​    lame-3.100           | 496 KB    | ##################################### | 100% [A[A[A[A[A[A[A[A[A[A[A​    
​    
​    
​    
​    
​    
​    
​    
​    lame-3.100           | 496 KB    | ##################################### | 100% [A[A[A[A[A[A[A[A[A[A[A​    
​    
​    
​    
​    
​    
​    
​    
​    
​    certifi-2024.7.4     | 156 KB    | ##################################### | 100% [A[A[A[A[A[A[A[A[A[A[A[A​    
​    
​    
​    
​    
​    
​    
​    
​    
​    certifi-2024.7.4     | 156 KB    | ##################################### | 100% [A[A[A[A[A[A[A[A[A[A[A[A​    
​    
​    
​    
​    
​    
​    
​    
​    
​    
​    ca-certificates-2024 | 151 KB    | ##################################### | 100% [A[A[A[A[A[A[A[A[A[A[A[A[A​    
​    
​    
​    
​    
​    
​    
​    
​    
​    
​    ca-certificates-2024 | 151 KB    | ##################################### | 100% [A[A[A[A[A[A[A[A[A[A[A[A[A​    
​    
​    
​    
​    
​    
​    
​    
​    
​    
​    
​    filelock-3.15.4      | 17 KB     | ##################################### | 100% [A[A[A[A[A[A[A[A[A[A[A[A[A[A​    
​    
​    
​    
​    
​    
​    
​    
​    
​    
​    
​    filelock-3.15.4      | 17 KB     | ##################################### | 100% [A[A[A[A[A[A[A[A[A[A[A[A[A[A​    
​    
​    
​    
​    
​    
​    
​    
​    
​    
​    
​    
​    pytorch-mutex-1.0    | 3 KB      | ##################################### | 100% [A[A[A[A[A[A[A[A[A[A[A[A[A[A[A​    
​    
​    
​    
​    
​    
​    
​    
​    
​    
​    
​    
​    
​    cpuonly-2.0          | 2 KB      | ##################################### | 100% [A[A[A[A[A[A[A[A[A[A[A[A[A[A[A[Apytorch-2.3.0        | 31.5 MB   | ##############################        |  81% [A​    
​    torchaudio-2.3.0     | 5.1 MB    | ##################################### | 100% [A[A[A[A[A​    
​    
​    torchaudio-2.3.0     | 5.1 MB    | ##################################### | 100% [A[A[A[A[Apytorch-2.3.0        | 31.5 MB   | ################################7     |  88% [A​    
​    
​    
​    gnutls-3.6.13        | 2.0 MB    | ##################################### | 100% [A[A[A[A[A[A[A​    
​    
​    
​    
​    libtorch-2.3.0       | 47.7 MB   | ##################################### | 100% [A[A[A[A[A[A[Apytorch-2.3.0        | 31.5 MB   | ##################################### | 100% [A[A[A[A[A[A[A[A​    
​    [A[A[A[A​    
​    
​    [A[A[A[A[A​    
​    
​    
​    [A[A[A[A[A[A​    
​    
​    
​    
​    [A[A[A[A[A[A[A​    
​    
​    
​    
​    
​    [A[A[A[A[A[A[A[A​    
​    
​    
​    
​    
​    
​    [A[A[A[A[A[A[A[A[A​    
​    
​    
​    
​    
​    
​    
​    [A[A[A[A[A[A[A[A[A[A​    
​    
​    
​    
​    
​    
​    
​    
​    [A[A[A[A[A[A[A[A[A[A[A​    
​    
​    
​    
​    
​    
​    
​    
​    
​    [A[A[A[A[A[A[A[A[A[A[A[A​    
​    
​    
​    
​    
​    
​    
​    
​    
​    
​    [A[A[A[A[A[A[A[A[A[A[A[A[A​    
​    
​    
​    
​    
​    
​    
​    
​    
​    
​    
​    [A[A[A[A[A[A[A[A[A[A[A[A[A[A​    
​    
​    
​    
​    
​    
​    
​    
​    
​    
​    
​    
​    [A[A[A[A[A[A[A[A[A[A[A[A[A[A[A​    
​    
​    
​    
​    
​    
​    
​    
​    
​    
​    
​    
​    
​    [A[A[A[A[A[A[A[A[A[A[A[A[A[A[A[APreparing transaction: doneVerifying transaction: doneExecuting transaction: doneLooking in indexes: https://mirrors.cloud.tencent.com/pypi/simpleCollecting cmakeDownloading https://mirrors.cloud.tencent.com/pypi/packages/78/5e/c274ffd124b8d4d95734af94c1080f0421c89dabdea2475651a7bd1e02ca/cmake-3.30.1-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (26.9 MB)[2K     [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m26.9/26.9 MB[0m [31m44.5 MB/s[0m eta [36m0:00:00[0m00:01[0m00:01[0m[?25hInstalling collected packages: cmakeSuccessfully installed cmake-3.30.1Looking in indexes: https://mirrors.cloud.tencent.com/pypi/simpleCollecting ninjaDownloading https://mirrors.cloud.tencent.com/pypi/packages/6d/92/8d7aebd4430ab5ff65df2bfee6d5745f95c004284db2d8ca76dcbfd9de47/ninja-1.11.1.1-py2.py3-none-manylinux1_x86_64.manylinux_2_5_x86_64.whl (307 kB)[2K     [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m307.2/307.2 kB[0m [31m5.0 MB/s[0m eta [36m0:00:00[0ma [36m0:00:01[0m[?25hInstalling collected packages: ninjaSuccessfully installed ninja-1.11.1.1Looking in indexes: https://mirrors.cloud.tencent.com/pypi/simpleCollecting neural_speedDownloading https://mirrors.cloud.tencent.com/pypi/packages/de/52/ce34338c918dafeb6bd9d777ce4a0b75f814fe4eedfea4bd6b13f53efbd1/neural_speed-1.0-cp311-cp311-manylinux_2_28_x86_64.whl (23.3 MB)[2K     [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m23.3/23.3 MB[0m [31m8.2 MB/s[0m eta [36m0:00:00[0m00:01[0m00:01[0m[?25hInstalling collected packages: neural_speedSuccessfully installed neural_speed-1.0Looking in indexes: https://mirrors.cloud.tencent.com/pypi/simpleCollecting intel-extension-for-transformersDownloading https://mirrors.cloud.tencent.com/pypi/packages/ce/73/8ab583a1dec951684e42b71fd0058c1c9bfc7ae59c42f741d6e698bcf978/intel_extension_for_transformers-1.4.2-cp311-cp311-manylinux_2_28_x86_64.whl (45.3 MB)[2K     [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m45.3/45.3 MB[0m [31m9.7 MB/s[0m eta [36m0:00:00[0m:00:01[0m00:01[0m[?25hRequirement already satisfied: packaging in /opt/conda/lib/python3.11/site-packages (from intel-extension-for-transformers) (24.0)Requirement already satisfied: numpy in /opt/conda/lib/python3.11/site-packages (from intel-extension-for-transformers) (1.26.4)Collecting schema (from intel-extension-for-transformers)Downloading https://mirrors.cloud.tencent.com/pypi/packages/ad/1b/81855a88c6db2b114d5b2e9f96339190d5ee4d1b981d217fa32127bb00e0/schema-0.7.7-py2.py3-none-any.whl (18 kB)Requirement already satisfied: pyyaml in /opt/conda/lib/python3.11/site-packages (from intel-extension-for-transformers) (6.0.1)Collecting neural-compressor (from intel-extension-for-transformers)Downloading https://mirrors.cloud.tencent.com/pypi/packages/c2/14/0a218e67d03d7da680321552417631b3ac9f9e0d47b38d400da8dd1d41f1/neural_compressor-2.6-py3-none-any.whl (1.5 MB)[2K     [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m1.5/1.5 MB[0m [31m9.6 MB/s[0m eta [36m0:00:00[0m:00:01[0m00:01[0m[?25hCollecting transformers (from intel-extension-for-transformers)Downloading https://mirrors.cloud.tencent.com/pypi/packages/e3/89/66b0d61558c971dd2c8cbe125a471603fce0a1b8850c2f4d99a07584fca2/transformers-4.43.1-py3-none-any.whl (9.4 MB)[2K     [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m9.4/9.4 MB[0m [31m9.5 MB/s[0m eta [36m0:00:00[0m00:01[0m00:01[0mm[?25hCollecting deprecated>=1.2.13 (from neural-compressor->intel-extension-for-transformers)Downloading https://mirrors.cloud.tencent.com/pypi/packages/20/8d/778b7d51b981a96554f29136cd59ca7880bf58094338085bcf2a979a0e6a/Deprecated-1.2.14-py2.py3-none-any.whl (9.6 kB)Collecting opencv-python-headless (from neural-compressor->intel-extension-for-transformers)Downloading https://mirrors.cloud.tencent.com/pypi/packages/d1/09/248f86a404567303cdf120e4a301f389b68e3b18e5c0cc428de327da609c/opencv_python_headless-4.10.0.84-cp37-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (49.9 MB)[2K     [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m49.9/49.9 MB[0m [31m8.1 MB/s[0m eta [36m0:00:00[0m00:01[0m00:01[0mm[?25hRequirement already satisfied: pandas in /opt/conda/lib/python3.11/site-packages (from neural-compressor->intel-extension-for-transformers) (2.2.1)Requirement already satisfied: Pillow in /opt/conda/lib/python3.11/site-packages (from neural-compressor->intel-extension-for-transformers) (10.2.0)Collecting prettytable (from neural-compressor->intel-extension-for-transformers)Downloading https://mirrors.cloud.tencent.com/pypi/packages/c5/16/ec5cc65437dce97d2814a7ba31842b0ee958d102f6e99e264c35f15c328f/prettytable-3.10.2-py3-none-any.whl (28 kB)Requirement already satisfied: psutil in /opt/conda/lib/python3.11/site-packages (from neural-compressor->intel-extension-for-transformers) (5.9.8)Requirement already satisfied: py-cpuinfo in /opt/conda/lib/python3.11/site-packages (from neural-compressor->intel-extension-for-transformers) (9.0.0)Requirement already satisfied: requests in /opt/conda/lib/python3.11/site-packages (from neural-compressor->intel-extension-for-transformers) (2.31.0)Requirement already satisfied: scikit-learn in /opt/conda/lib/python3.11/site-packages (from neural-compressor->intel-extension-for-transformers) (1.4.1.post1)Collecting pycocotools (from neural-compressor->intel-extension-for-transformers)Downloading https://mirrors.cloud.tencent.com/pypi/packages/8b/d4/7279d072c0255d07c541326f6058effb1b08190f49695bf2c22aae666878/pycocotools-2.0.8-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (458 kB)[2K     [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m458.7/458.7 kB[0m [31m10.6 MB/s[0m eta [36m0:00:00[0ma [36m0:00:01[0m[?25hRequirement already satisfied: filelock in /opt/conda/lib/python3.11/site-packages (from transformers->intel-extension-for-transformers) (3.15.4)Collecting huggingface-hub<1.0,>=0.23.2 (from transformers->intel-extension-for-transformers)Downloading https://mirrors.cloud.tencent.com/pypi/packages/96/e6/a1fd9cccd2c08244243aeef71b61cb9b2ba26575d8fd6f7c41edc95e9de0/huggingface_hub-0.24.1-py3-none-any.whl (417 kB)[2K     [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m417.2/417.2 kB[0m [31m1.7 MB/s[0m eta [36m0:00:00[0m00:01[0m00:01[0m[?25hCollecting regex!=2019.12.17 (from transformers->intel-extension-for-transformers)Downloading https://mirrors.cloud.tencent.com/pypi/packages/39/29/8158a6e69e97b9c72fab0b46fe4d57c789d07ef91fe4afde23721e7cac61/regex-2024.5.15-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (785 kB)[2K     [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m785.0/785.0 kB[0m [31m776.1 kB/s[0m eta [36m0:00:00[0m00:01[0m00:01[0m[?25hCollecting tokenizers<0.20,>=0.19 (from transformers->intel-extension-for-transformers)Downloading https://mirrors.cloud.tencent.com/pypi/packages/a7/03/fb50fc03f86016b227a967c8d474f90230c885c0d18f78acdfda7a96ce56/tokenizers-0.19.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB)[2K     [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m3.6/3.6 MB[0m [31m9.4 MB/s[0m eta [36m0:00:00[0m00:01[0mm00:01[0m[?25hCollecting safetensors>=0.4.1 (from transformers->intel-extension-for-transformers)Downloading https://mirrors.cloud.tencent.com/pypi/packages/d5/85/1e7d2804cbf82204cde462d16f1cb0ff5814b03f559fb46ceaa6b7020db4/safetensors-0.4.3-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.2 MB)[2K     [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m1.2/1.2 MB[0m [31m9.8 MB/s[0m eta [36m0:00:00[0mta [36m0:00:01[0m[?25hRequirement already satisfied: tqdm>=4.27 in /opt/conda/lib/python3.11/site-packages (from transformers->intel-extension-for-transformers) (4.66.2)Collecting wrapt<2,>=1.10 (from deprecated>=1.2.13->neural-compressor->intel-extension-for-transformers)Downloading https://mirrors.cloud.tencent.com/pypi/packages/6e/52/2da48b35193e39ac53cfb141467d9f259851522d0e8c87153f0ba4205fb1/wrapt-1.16.0-cp311-cp311-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (80 kB)[2K     [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m80.7/80.7 kB[0m [31m9.7 MB/s[0m eta [36m0:00:00[0m[?25hRequirement already satisfied: fsspec>=2023.5.0 in /opt/conda/lib/python3.11/site-packages (from huggingface-hub<1.0,>=0.23.2->transformers->intel-extension-for-transformers) (2024.3.1)Requirement already satisfied: typing-extensions>=3.7.4.3 in /opt/conda/lib/python3.11/site-packages (from huggingface-hub<1.0,>=0.23.2->transformers->intel-extension-for-transformers) (4.10.0)Requirement already satisfied: python-dateutil>=2.8.2 in /opt/conda/lib/python3.11/site-packages (from pandas->neural-compressor->intel-extension-for-transformers) (2.9.0)Requirement already satisfied: pytz>=2020.1 in /opt/conda/lib/python3.11/site-packages (from pandas->neural-compressor->intel-extension-for-transformers) (2024.1)Requirement already satisfied: tzdata>=2022.7 in /opt/conda/lib/python3.11/site-packages (from pandas->neural-compressor->intel-extension-for-transformers) (2024.1)Requirement already satisfied: wcwidth in /opt/conda/lib/python3.11/site-packages (from prettytable->neural-compressor->intel-extension-for-transformers) (0.2.13)Requirement already satisfied: matplotlib>=2.1.0 in /opt/conda/lib/python3.11/site-packages (from pycocotools->neural-compressor->intel-extension-for-transformers) (3.8.3)Requirement already satisfied: charset-normalizer<4,>=2 in /opt/conda/lib/python3.11/site-packages (from requests->neural-compressor->intel-extension-for-transformers) (3.3.2)Requirement already satisfied: idna<4,>=2.5 in /opt/conda/lib/python3.11/site-packages (from requests->neural-compressor->intel-extension-for-transformers) (3.6)Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/conda/lib/python3.11/site-packages (from requests->neural-compressor->intel-extension-for-transformers) (2.2.1)Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.11/site-packages (from requests->neural-compressor->intel-extension-for-transformers) (2024.7.4)Requirement already satisfied: scipy>=1.6.0 in /opt/conda/lib/python3.11/site-packages (from scikit-learn->neural-compressor->intel-extension-for-transformers) (1.12.0)Requirement already satisfied: joblib>=1.2.0 in /opt/conda/lib/python3.11/site-packages (from scikit-learn->neural-compressor->intel-extension-for-transformers) (1.3.2)Requirement already satisfied: threadpoolctl>=2.0.0 in /opt/conda/lib/python3.11/site-packages (from scikit-learn->neural-compressor->intel-extension-for-transformers) (3.4.0)Requirement already satisfied: contourpy>=1.0.1 in /opt/conda/lib/python3.11/site-packages (from matplotlib>=2.1.0->pycocotools->neural-compressor->intel-extension-for-transformers) (1.2.0)Requirement already satisfied: cycler>=0.10 in /opt/conda/lib/python3.11/site-packages (from matplotlib>=2.1.0->pycocotools->neural-compressor->intel-extension-for-transformers) (0.12.1)Requirement already satisfied: fonttools>=4.22.0 in /opt/conda/lib/python3.11/site-packages (from matplotlib>=2.1.0->pycocotools->neural-compressor->intel-extension-for-transformers) (4.50.0)Requirement already satisfied: kiwisolver>=1.3.1 in /opt/conda/lib/python3.11/site-packages (from matplotlib>=2.1.0->pycocotools->neural-compressor->intel-extension-for-transformers) (1.4.5)Requirement already satisfied: pyparsing>=2.3.1 in /opt/conda/lib/python3.11/site-packages (from matplotlib>=2.1.0->pycocotools->neural-compressor->intel-extension-for-transformers) (3.1.2)Requirement already satisfied: six>=1.5 in /opt/conda/lib/python3.11/site-packages (from python-dateutil>=2.8.2->pandas->neural-compressor->intel-extension-for-transformers) (1.16.0)Installing collected packages: schema, wrapt, safetensors, regex, prettytable, opencv-python-headless, huggingface-hub, deprecated, tokenizers, pycocotools, transformers, neural-compressor, intel-extension-for-transformersSuccessfully installed deprecated-1.2.14 huggingface-hub-0.24.1 intel-extension-for-transformers-1.4.2 neural-compressor-2.6 opencv-python-headless-4.10.0.84 prettytable-3.10.2 pycocotools-2.0.8 regex-2024.5.15 safetensors-0.4.3 schema-0.7.7 tokenizers-0.19.1 transformers-4.43.1 wrapt-1.16.0Looking in indexes: https://mirrors.cloud.tencent.com/pypi/simpleCollecting modelscopeDownloading https://mirrors.cloud.tencent.com/pypi/packages/38/37/9fe505ebc67ba5e0345a69d6e8b2ee8630523975b484d221691ef60182bd/modelscope-1.16.1-py3-none-any.whl (5.7 MB)[2K     [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m5.7/5.7 MB[0m [31m4.1 MB/s[0m eta [36m0:00:00[0m00:01[0m00:01[0m[?25hRequirement already satisfied: requests>=2.25 in /opt/conda/lib/python3.11/site-packages (from modelscope) (2.31.0)Requirement already satisfied: tqdm>=4.64.0 in /opt/conda/lib/python3.11/site-packages (from modelscope) (4.66.2)Requirement already satisfied: urllib3>=1.26 in /opt/conda/lib/python3.11/site-packages (from modelscope) (2.2.1)Requirement already satisfied: charset-normalizer<4,>=2 in /opt/conda/lib/python3.11/site-packages (from requests>=2.25->modelscope) (3.3.2)Requirement already satisfied: idna<4,>=2.5 in /opt/conda/lib/python3.11/site-packages (from requests>=2.25->modelscope) (3.6)Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.11/site-packages (from requests>=2.25->modelscope) (2024.7.4)Installing collected packages: modelscopeSuccessfully installed modelscope-1.16.1Looking in indexes: https://mirrors.cloud.tencent.com/pypi/simpleRequirement already satisfied: transformers in /opt/conda/lib/python3.11/site-packages (4.43.1)Requirement already satisfied: filelock in /opt/conda/lib/python3.11/site-packages (from transformers) (3.15.4)Requirement already satisfied: huggingface-hub<1.0,>=0.23.2 in /opt/conda/lib/python3.11/site-packages (from transformers) (0.24.1)Requirement already satisfied: numpy>=1.17 in /opt/conda/lib/python3.11/site-packages (from transformers) (1.26.4)Requirement already satisfied: packaging>=20.0 in /opt/conda/lib/python3.11/site-packages (from transformers) (24.0)Requirement already satisfied: pyyaml>=5.1 in /opt/conda/lib/python3.11/site-packages (from transformers) (6.0.1)Requirement already satisfied: regex!=2019.12.17 in /opt/conda/lib/python3.11/site-packages (from transformers) (2024.5.15)Requirement already satisfied: requests in /opt/conda/lib/python3.11/site-packages (from transformers) (2.31.0)Requirement already satisfied: tokenizers<0.20,>=0.19 in /opt/conda/lib/python3.11/site-packages (from transformers) (0.19.1)Requirement already satisfied: safetensors>=0.4.1 in /opt/conda/lib/python3.11/site-packages (from transformers) (0.4.3)Requirement already satisfied: tqdm>=4.27 in /opt/conda/lib/python3.11/site-packages (from transformers) (4.66.2)Requirement already satisfied: fsspec>=2023.5.0 in /opt/conda/lib/python3.11/site-packages (from huggingface-hub<1.0,>=0.23.2->transformers) (2024.3.1)Requirement already satisfied: typing-extensions>=3.7.4.3 in /opt/conda/lib/python3.11/site-packages (from huggingface-hub<1.0,>=0.23.2->transformers) (4.10.0)Requirement already satisfied: charset-normalizer<4,>=2 in /opt/conda/lib/python3.11/site-packages (from requests->transformers) (3.3.2)Requirement already satisfied: idna<4,>=2.5 in /opt/conda/lib/python3.11/site-packages (from requests->transformers) (3.6)Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/conda/lib/python3.11/site-packages (from requests->transformers) (2.2.1)Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.11/site-packages (from requests->transformers) (2024.7.4)Looking in indexes: https://mirrors.cloud.tencent.com/pypi/simpleRequirement already satisfied: pyOpenSSL in /opt/conda/lib/python3.11/site-packages (24.0.0)Collecting pyOpenSSLDownloading https://mirrors.cloud.tencent.com/pypi/packages/d9/dd/e0aa7ebef5168c75b772eda64978c597a9129b46be17779054652a7999e4/pyOpenSSL-24.2.1-py3-none-any.whl (58 kB)[2K     [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m58.4/58.4 kB[0m [31m1.7 MB/s[0m eta [36m0:00:00[0m[?25hRequirement already satisfied: cryptography<44,>=41.0.5 in /opt/conda/lib/python3.11/site-packages (from pyOpenSSL) (42.0.5)Requirement already satisfied: cffi>=1.12 in /opt/conda/lib/python3.11/site-packages (from cryptography<44,>=41.0.5->pyOpenSSL) (1.16.0)Requirement already satisfied: pycparser in /opt/conda/lib/python3.11/site-packages (from cffi>=1.12->cryptography<44,>=41.0.5->pyOpenSSL) (2.22)Installing collected packages: pyOpenSSLAttempting uninstall: pyOpenSSLFound existing installation: pyOpenSSL 24.0.0Uninstalling pyOpenSSL-24.0.0:Successfully uninstalled pyOpenSSL-24.0.0Successfully installed pyOpenSSL-24.2.1Looking in indexes: https://mirrors.cloud.tencent.com/pypi/simpleCollecting sentencepieceDownloading https://mirrors.cloud.tencent.com/pypi/packages/fb/12/2f5c8d4764b00033cf1c935b702d3bb878d10be9f0b87f0253495832d85f/sentencepiece-0.2.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)[2K     [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m1.3/1.3 MB[0m [31m10.1 MB/s[0m eta [36m0:00:00[0ma [36m0:00:01[0m[?25hInstalling collected packages: sentencepieceSuccessfully installed sentencepiece-0.2.0Looking in indexes: https://mirrors.cloud.tencent.com/pypi/simpleCollecting xformersDownloading https://mirrors.cloud.tencent.com/pypi/packages/38/c7/e0d61e0a59536e52634a5939c7c3556ec1a52f51b6b4e8eeeed16702a2c2/xformers-0.0.27-cp311-cp311-manylinux2014_x86_64.whl (164.2 MB)[2K     [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m164.2/164.2 MB[0m [31m4.5 MB/s[0m eta [36m0:00:00[0m00:01[0m00:01[0m[?25hRequirement already satisfied: numpy in /opt/conda/lib/python3.11/site-packages (from xformers) (1.26.4)Collecting torch==2.3.1 (from xformers)Downloading https://mirrors.cloud.tencent.com/pypi/packages/07/9a/4c5e74264439837814656201da13a898056a5201c976ef042544bceb840f/torch-2.3.1-cp311-cp311-manylinux1_x86_64.whl (779.2 MB)[2K     [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m779.2/779.2 MB[0m [31m2.2 MB/s[0m eta [36m0:00:00[0m00:01[0m00:05[0m[?25hRequirement already satisfied: filelock in /opt/conda/lib/python3.11/site-packages (from torch==2.3.1->xformers) (3.15.4)Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/lib/python3.11/site-packages (from torch==2.3.1->xformers) (4.10.0)Requirement already satisfied: sympy in /opt/conda/lib/python3.11/site-packages (from torch==2.3.1->xformers) (1.12)Requirement already satisfied: networkx in /opt/conda/lib/python3.11/site-packages (from torch==2.3.1->xformers) (3.2.1)Requirement already satisfied: jinja2 in /opt/conda/lib/python3.11/site-packages (from torch==2.3.1->xformers) (3.1.3)Requirement already satisfied: fsspec in /opt/conda/lib/python3.11/site-packages (from torch==2.3.1->xformers) (2024.3.1)Collecting nvidia-cuda-nvrtc-cu12==12.1.105 (from torch==2.3.1->xformers)Downloading https://mirrors.cloud.tencent.com/pypi/packages/b6/9f/c64c03f49d6fbc56196664d05dba14e3a561038a81a638eeb47f4d4cfd48/nvidia_cuda_nvrtc_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (23.7 MB)[2K     [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m23.7/23.7 MB[0m [31m10.7 MB/s[0m eta [36m0:00:00[0m00:01[0m00:01[0m[?25hCollecting nvidia-cuda-runtime-cu12==12.1.105 (from torch==2.3.1->xformers)Downloading https://mirrors.cloud.tencent.com/pypi/packages/eb/d5/c68b1d2cdfcc59e72e8a5949a37ddb22ae6cade80cd4a57a84d4c8b55472/nvidia_cuda_runtime_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (823 kB)[2K     [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m823.6/823.6 kB[0m [31m2.1 MB/s[0m eta [36m0:00:00[0m00:01[0m00:01[0m[?25hCollecting nvidia-cuda-cupti-cu12==12.1.105 (from torch==2.3.1->xformers)Downloading https://mirrors.cloud.tencent.com/pypi/packages/7e/00/6b218edd739ecfc60524e585ba8e6b00554dd908de2c9c66c1af3e44e18d/nvidia_cuda_cupti_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (14.1 MB)[2K     [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m14.1/14.1 MB[0m [31m11.9 MB/s[0m eta [36m0:00:00[0ma [36m0:00:01[0m[?25hCollecting nvidia-cudnn-cu12==8.9.2.26 (from torch==2.3.1->xformers)Downloading https://mirrors.cloud.tencent.com/pypi/packages/ff/74/a2e2be7fb83aaedec84f391f082cf765dfb635e7caa9b49065f73e4835d8/nvidia_cudnn_cu12-8.9.2.26-py3-none-manylinux1_x86_64.whl (731.7 MB)[2K     [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m731.7/731.7 MB[0m [31m2.4 MB/s[0m eta [36m0:00:00[0m00:01[0m00:02[0m[?25hCollecting nvidia-cublas-cu12==12.1.3.1 (from torch==2.3.1->xformers)Downloading https://mirrors.cloud.tencent.com/pypi/packages/37/6d/121efd7382d5b0284239f4ab1fc1590d86d34ed4a4a2fdb13b30ca8e5740/nvidia_cublas_cu12-12.1.3.1-py3-none-manylinux1_x86_64.whl (410.6 MB)[2K     [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m410.6/410.6 MB[0m [31m4.1 MB/s[0m eta [36m0:00:00[0m00:01[0m00:02[0m[?25hCollecting nvidia-cufft-cu12==11.0.2.54 (from torch==2.3.1->xformers)Downloading https://mirrors.cloud.tencent.com/pypi/packages/86/94/eb540db023ce1d162e7bea9f8f5aa781d57c65aed513c33ee9a5123ead4d/nvidia_cufft_cu12-11.0.2.54-py3-none-manylinux1_x86_64.whl (121.6 MB)[2K     [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m121.6/121.6 MB[0m [31m8.7 MB/s[0m eta [36m0:00:00[0m00:01[0m00:01[0m[?25hCollecting nvidia-curand-cu12==10.3.2.106 (from torch==2.3.1->xformers)Downloading https://mirrors.cloud.tencent.com/pypi/packages/44/31/4890b1c9abc496303412947fc7dcea3d14861720642b49e8ceed89636705/nvidia_curand_cu12-10.3.2.106-py3-none-manylinux1_x86_64.whl (56.5 MB)[2K     [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m56.5/56.5 MB[0m [31m6.7 MB/s[0m eta [36m0:00:00[0m00:01[0m00:01[0mm[?25hCollecting nvidia-cusolver-cu12==11.4.5.107 (from torch==2.3.1->xformers)Downloading https://mirrors.cloud.tencent.com/pypi/packages/bc/1d/8de1e5c67099015c834315e333911273a8c6aaba78923dd1d1e25fc5f217/nvidia_cusolver_cu12-11.4.5.107-py3-none-manylinux1_x86_64.whl (124.2 MB)[2K     [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m124.2/124.2 MB[0m [31m8.7 MB/s[0m eta [36m0:00:00[0m00:01[0m00:01[0m[?25hCollecting nvidia-cusparse-cu12==12.1.0.106 (from torch==2.3.1->xformers)Downloading https://mirrors.cloud.tencent.com/pypi/packages/65/5b/cfaeebf25cd9fdec14338ccb16f6b2c4c7fa9163aefcf057d86b9cc248bb/nvidia_cusparse_cu12-12.1.0.106-py3-none-manylinux1_x86_64.whl (196.0 MB)[2K     [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m196.0/196.0 MB[0m [31m7.7 MB/s[0m eta [36m0:00:00[0m00:01[0m00:01[0m[?25hCollecting nvidia-nccl-cu12==2.20.5 (from torch==2.3.1->xformers)Downloading https://mirrors.cloud.tencent.com/pypi/packages/4b/2a/0a131f572aa09f741c30ccd45a8e56316e8be8dfc7bc19bf0ab7cfef7b19/nvidia_nccl_cu12-2.20.5-py3-none-manylinux2014_x86_64.whl (176.2 MB)[2K     [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m176.2/176.2 MB[0m [31m8.5 MB/s[0m eta [36m0:00:00[0m00:01[0m00:01[0m[?25hCollecting nvidia-nvtx-cu12==12.1.105 (from torch==2.3.1->xformers)Downloading https://mirrors.cloud.tencent.com/pypi/packages/da/d3/8057f0587683ed2fcd4dbfbdfdfa807b9160b809976099d36b8f60d08f03/nvidia_nvtx_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (99 kB)[2K     [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m99.1/99.1 kB[0m [31m9.8 MB/s[0m eta [36m0:00:00[0m[?25hCollecting triton==2.3.1 (from torch==2.3.1->xformers)Downloading https://mirrors.cloud.tencent.com/pypi/packages/64/16/956b7b9d2ed3a437a1a06792b2ae2e3c49147296ba2f4d59fcee376ded8f/triton-2.3.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (168.1 MB)[2K     [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m168.1/168.1 MB[0m [31m22.6 MB/s[0m eta [36m0:00:00[0m00:01[0m00:01[0m[?25hCollecting nvidia-nvjitlink-cu12 (from nvidia-cusolver-cu12==11.4.5.107->torch==2.3.1->xformers)Downloading https://mirrors.cloud.tencent.com/pypi/packages/75/bc/e0d0dbb85246a086ab14839979039647bce501d8c661a159b8b019d987b7/nvidia_nvjitlink_cu12-12.5.82-py3-none-manylinux2014_x86_64.whl (21.3 MB)[2K     [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m21.3/21.3 MB[0m [31m9.1 MB/s[0m eta [36m0:00:00[0m00:01[0mm00:01[0m[?25hRequirement already satisfied: MarkupSafe>=2.0 in /opt/conda/lib/python3.11/site-packages (from jinja2->torch==2.3.1->xformers) (2.1.5)Requirement already satisfied: mpmath>=0.19 in /opt/conda/lib/python3.11/site-packages (from sympy->torch==2.3.1->xformers) (1.3.0)Installing collected packages: triton, nvidia-nvtx-cu12, nvidia-nvjitlink-cu12, nvidia-nccl-cu12, nvidia-curand-cu12, nvidia-cufft-cu12, nvidia-cuda-runtime-cu12, nvidia-cuda-nvrtc-cu12, nvidia-cuda-cupti-cu12, nvidia-cublas-cu12, nvidia-cusparse-cu12, nvidia-cudnn-cu12, nvidia-cusolver-cu12, torch, xformersAttempting uninstall: torchFound existing installation: torch 2.3.0Uninstalling torch-2.3.0:Successfully uninstalled torch-2.3.0Successfully installed nvidia-cublas-cu12-12.1.3.1 nvidia-cuda-cupti-cu12-12.1.105 nvidia-cuda-nvrtc-cu12-12.1.105 nvidia-cuda-runtime-cu12-12.1.105 nvidia-cudnn-cu12-8.9.2.26 nvidia-cufft-cu12-11.0.2.54 nvidia-curand-cu12-10.3.2.106 nvidia-cusolver-cu12-11.4.5.107 nvidia-cusparse-cu12-12.1.0.106 nvidia-nccl-cu12-2.20.5 nvidia-nvjitlink-cu12-12.5.82 nvidia-nvtx-cu12-12.1.105 torch-2.3.1 triton-2.3.1 xformers-0.0.27Looking in indexes: https://mirrors.cloud.tencent.com/pypi/simpleCollecting accelerateDownloading https://mirrors.cloud.tencent.com/pypi/packages/15/33/b6b4ad5efa8b9f4275d4ed17ff8a44c97276171341ba565fdffb0e3dc5e8/accelerate-0.33.0-py3-none-any.whl (315 kB)[2K     [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m315.1/315.1 kB[0m [31m4.1 MB/s[0m eta [36m0:00:00[0m00:01[0m00:01[0m[?25hRequirement already satisfied: numpy<2.0.0,>=1.17 in /opt/conda/lib/python3.11/site-packages (from accelerate) (1.26.4)Requirement already satisfied: packaging>=20.0 in /opt/conda/lib/python3.11/site-packages (from accelerate) (24.0)Requirement already satisfied: psutil in /opt/conda/lib/python3.11/site-packages (from accelerate) (5.9.8)Requirement already satisfied: pyyaml in /opt/conda/lib/python3.11/site-packages (from accelerate) (6.0.1)Requirement already satisfied: torch>=1.10.0 in /opt/conda/lib/python3.11/site-packages (from accelerate) (2.3.1)Requirement already satisfied: huggingface-hub>=0.21.0 in /opt/conda/lib/python3.11/site-packages (from accelerate) (0.24.1)Requirement already satisfied: safetensors>=0.3.1 in /opt/conda/lib/python3.11/site-packages (from accelerate) (0.4.3)Requirement already satisfied: filelock in /opt/conda/lib/python3.11/site-packages (from huggingface-hub>=0.21.0->accelerate) (3.15.4)Requirement already satisfied: fsspec>=2023.5.0 in /opt/conda/lib/python3.11/site-packages (from huggingface-hub>=0.21.0->accelerate) (2024.3.1)Requirement already satisfied: requests in /opt/conda/lib/python3.11/site-packages (from huggingface-hub>=0.21.0->accelerate) (2.31.0)Requirement already satisfied: tqdm>=4.42.1 in /opt/conda/lib/python3.11/site-packages (from huggingface-hub>=0.21.0->accelerate) (4.66.2)Requirement already satisfied: typing-extensions>=3.7.4.3 in /opt/conda/lib/python3.11/site-packages (from huggingface-hub>=0.21.0->accelerate) (4.10.0)Requirement already satisfied: sympy in /opt/conda/lib/python3.11/site-packages (from torch>=1.10.0->accelerate) (1.12)Requirement already satisfied: networkx in /opt/conda/lib/python3.11/site-packages (from torch>=1.10.0->accelerate) (3.2.1)Requirement already satisfied: jinja2 in /opt/conda/lib/python3.11/site-packages (from torch>=1.10.0->accelerate) (3.1.3)Requirement already satisfied: nvidia-cuda-nvrtc-cu12==12.1.105 in /opt/conda/lib/python3.11/site-packages (from torch>=1.10.0->accelerate) (12.1.105)Requirement already satisfied: nvidia-cuda-runtime-cu12==12.1.105 in /opt/conda/lib/python3.11/site-packages (from torch>=1.10.0->accelerate) (12.1.105)Requirement already satisfied: nvidia-cuda-cupti-cu12==12.1.105 in /opt/conda/lib/python3.11/site-packages (from torch>=1.10.0->accelerate) (12.1.105)Requirement already satisfied: nvidia-cudnn-cu12==8.9.2.26 in /opt/conda/lib/python3.11/site-packages (from torch>=1.10.0->accelerate) (8.9.2.26)Requirement already satisfied: nvidia-cublas-cu12==12.1.3.1 in /opt/conda/lib/python3.11/site-packages (from torch>=1.10.0->accelerate) (12.1.3.1)Requirement already satisfied: nvidia-cufft-cu12==11.0.2.54 in /opt/conda/lib/python3.11/site-packages (from torch>=1.10.0->accelerate) (11.0.2.54)Requirement already satisfied: nvidia-curand-cu12==10.3.2.106 in /opt/conda/lib/python3.11/site-packages (from torch>=1.10.0->accelerate) (10.3.2.106)Requirement already satisfied: nvidia-cusolver-cu12==11.4.5.107 in /opt/conda/lib/python3.11/site-packages (from torch>=1.10.0->accelerate) (11.4.5.107)Requirement already satisfied: nvidia-cusparse-cu12==12.1.0.106 in /opt/conda/lib/python3.11/site-packages (from torch>=1.10.0->accelerate) (12.1.0.106)Requirement already satisfied: nvidia-nccl-cu12==2.20.5 in /opt/conda/lib/python3.11/site-packages (from torch>=1.10.0->accelerate) (2.20.5)Requirement already satisfied: nvidia-nvtx-cu12==12.1.105 in /opt/conda/lib/python3.11/site-packages (from torch>=1.10.0->accelerate) (12.1.105)Requirement already satisfied: triton==2.3.1 in /opt/conda/lib/python3.11/site-packages (from torch>=1.10.0->accelerate) (2.3.1)Requirement already satisfied: nvidia-nvjitlink-cu12 in /opt/conda/lib/python3.11/site-packages (from nvidia-cusolver-cu12==11.4.5.107->torch>=1.10.0->accelerate) (12.5.82)Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/lib/python3.11/site-packages (from jinja2->torch>=1.10.0->accelerate) (2.1.5)Requirement already satisfied: charset-normalizer<4,>=2 in /opt/conda/lib/python3.11/site-packages (from requests->huggingface-hub>=0.21.0->accelerate) (3.3.2)Requirement already satisfied: idna<4,>=2.5 in /opt/conda/lib/python3.11/site-packages (from requests->huggingface-hub>=0.21.0->accelerate) (3.6)Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/conda/lib/python3.11/site-packages (from requests->huggingface-hub>=0.21.0->accelerate) (2.2.1)Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.11/site-packages (from requests->huggingface-hub>=0.21.0->accelerate) (2024.7.4)Requirement already satisfied: mpmath>=0.19 in /opt/conda/lib/python3.11/site-packages (from sympy->torch>=1.10.0->accelerate) (1.3.0)Installing collected packages: accelerateSuccessfully installed accelerate-0.33.0Looking in indexes: https://mirrors.cloud.tencent.com/pypi/simpleCollecting tiktokenDownloading https://mirrors.cloud.tencent.com/pypi/packages/61/b4/b80d1fe33015e782074e96bbbf4108ccd283b8deea86fb43c15d18b7c351/tiktoken-0.7.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.1 MB)[2K     [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m1.1/1.1 MB[0m [31m7.7 MB/s[0m eta [36m0:00:00[0m00:01[0m00:01[0m[?25hRequirement already satisfied: regex>=2022.1.18 in /opt/conda/lib/python3.11/site-packages (from tiktoken) (2024.5.15)Requirement already satisfied: requests>=2.26.0 in /opt/conda/lib/python3.11/site-packages (from tiktoken) (2.31.0)Requirement already satisfied: charset-normalizer<4,>=2 in /opt/conda/lib/python3.11/site-packages (from requests>=2.26.0->tiktoken) (3.3.2)Requirement already satisfied: idna<4,>=2.5 in /opt/conda/lib/python3.11/site-packages (from requests>=2.26.0->tiktoken) (3.6)Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/conda/lib/python3.11/site-packages (from requests>=2.26.0->tiktoken) (2.2.1)Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.11/site-packages (from requests>=2.26.0->tiktoken) (2024.7.4)Installing collected packages: tiktokenSuccessfully installed tiktoken-0.7.0Looking in indexes: https://mirrors.cloud.tencent.com/pypi/simpleCollecting transformers_stream_generatorDownloading https://mirrors.cloud.tencent.com/pypi/packages/42/c2/65f13aec253100e1916e9bd7965fe17bde796ebabeb1265f45191ab4ddc0/transformers-stream-generator-0.0.5.tar.gz (13 kB)Preparing metadata (setup.py) ... [?25ldone[?25hRequirement already satisfied: transformers>=4.26.1 in /opt/conda/lib/python3.11/site-packages (from transformers_stream_generator) (4.43.1)Requirement already satisfied: filelock in /opt/conda/lib/python3.11/site-packages (from transformers>=4.26.1->transformers_stream_generator) (3.15.4)Requirement already satisfied: huggingface-hub<1.0,>=0.23.2 in /opt/conda/lib/python3.11/site-packages (from transformers>=4.26.1->transformers_stream_generator) (0.24.1)Requirement already satisfied: numpy>=1.17 in /opt/conda/lib/python3.11/site-packages (from transformers>=4.26.1->transformers_stream_generator) (1.26.4)Requirement already satisfied: packaging>=20.0 in /opt/conda/lib/python3.11/site-packages (from transformers>=4.26.1->transformers_stream_generator) (24.0)Requirement already satisfied: pyyaml>=5.1 in /opt/conda/lib/python3.11/site-packages (from transformers>=4.26.1->transformers_stream_generator) (6.0.1)Requirement already satisfied: regex!=2019.12.17 in /opt/conda/lib/python3.11/site-packages (from transformers>=4.26.1->transformers_stream_generator) (2024.5.15)Requirement already satisfied: requests in /opt/conda/lib/python3.11/site-packages (from transformers>=4.26.1->transformers_stream_generator) (2.31.0)Requirement already satisfied: tokenizers<0.20,>=0.19 in /opt/conda/lib/python3.11/site-packages (from transformers>=4.26.1->transformers_stream_generator) (0.19.1)Requirement already satisfied: safetensors>=0.4.1 in /opt/conda/lib/python3.11/site-packages (from transformers>=4.26.1->transformers_stream_generator) (0.4.3)Requirement already satisfied: tqdm>=4.27 in /opt/conda/lib/python3.11/site-packages (from transformers>=4.26.1->transformers_stream_generator) (4.66.2)Requirement already satisfied: fsspec>=2023.5.0 in /opt/conda/lib/python3.11/site-packages (from huggingface-hub<1.0,>=0.23.2->transformers>=4.26.1->transformers_stream_generator) (2024.3.1)Requirement already satisfied: typing-extensions>=3.7.4.3 in /opt/conda/lib/python3.11/site-packages (from huggingface-hub<1.0,>=0.23.2->transformers>=4.26.1->transformers_stream_generator) (4.10.0)Requirement already satisfied: charset-normalizer<4,>=2 in /opt/conda/lib/python3.11/site-packages (from requests->transformers>=4.26.1->transformers_stream_generator) (3.3.2)Requirement already satisfied: idna<4,>=2.5 in /opt/conda/lib/python3.11/site-packages (from requests->transformers>=4.26.1->transformers_stream_generator) (3.6)Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/conda/lib/python3.11/site-packages (from requests->transformers>=4.26.1->transformers_stream_generator) (2.2.1)Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.11/site-packages (from requests->transformers>=4.26.1->transformers_stream_generator) (2024.7.4)Building wheels for collected packages: transformers_stream_generatorBuilding wheel for transformers_stream_generator (setup.py) ... [?25ldone[?25h  Created wheel for transformers_stream_generator: filename=transformers_stream_generator-0.0.5-py3-none-any.whl size=12425 sha256=85aa383732526ad8e749a82f5d4ce7076efa0ee456db4843096e1c118609740cStored in directory: /home/mw/.cache/pip/wheels/c0/9f/f6/f8573ca658852aa7cdde5a0e2717f767ac9b2dd19a7d2897b9Successfully built transformers_stream_generatorInstalling collected packages: transformers_stream_generatorSuccessfully installed transformers_stream_generator-0.0.5

# 在temp目录下进行模型在cpu上的转换存储
%cd ..
%cd temp

/home/mw
/home/mw/temp

!pip install datasets -i https://mirrors.cloud.tencent.com/pypi/simple
!pip install einops -i https://mirrors.cloud.tencent.com/pypi/simple

    Looking in indexes: https://mirrors.cloud.tencent.com/pypi/simpleCollecting datasetsDownloading https://mirrors.cloud.tencent.com/pypi/packages/60/2d/963b266bb8f88492d5ab4232d74292af8beb5b6fdae97902df9e284d4c32/datasets-2.20.0-py3-none-any.whl (547 kB)[2K     [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m547.8/547.8 kB[0m [31m2.3 MB/s[0m eta [36m0:00:00[0ma [36m0:00:01[0m[?25hRequirement already satisfied: filelock in /opt/conda/lib/python3.11/site-packages (from datasets) (3.15.4)Requirement already satisfied: numpy>=1.17 in /opt/conda/lib/python3.11/site-packages (from datasets) (1.26.4)Requirement already satisfied: pyarrow>=15.0.0 in /opt/conda/lib/python3.11/site-packages (from datasets) (15.0.2)Requirement already satisfied: pyarrow-hotfix in /opt/conda/lib/python3.11/site-packages (from datasets) (0.6)Requirement already satisfied: dill<0.3.9,>=0.3.0 in /opt/conda/lib/python3.11/site-packages (from datasets) (0.3.8)Requirement already satisfied: pandas in /opt/conda/lib/python3.11/site-packages (from datasets) (2.2.1)Collecting requests>=2.32.2 (from datasets)Downloading https://mirrors.cloud.tencent.com/pypi/packages/f9/9b/335f9764261e915ed497fcdeb11df5dfd6f7bf257d4a6a2a686d80da4d54/requests-2.32.3-py3-none-any.whl (64 kB)[2K     [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m64.9/64.9 kB[0m [31m4.0 MB/s[0m eta [36m0:00:00[0m[?25hCollecting tqdm>=4.66.3 (from datasets)Downloading https://mirrors.cloud.tencent.com/pypi/packages/18/eb/fdb7eb9e48b7b02554e1664afd3bd3f117f6b6d6c5881438a0b055554f9b/tqdm-4.66.4-py3-none-any.whl (78 kB)[2K     [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m78.3/78.3 kB[0m [31m5.3 MB/s[0m eta [36m0:00:00[0m[?25hCollecting xxhash (from datasets)Downloading https://mirrors.cloud.tencent.com/pypi/packages/eb/3a/25c4aecb61a49d4415fd71d4f66a8a5b558dd44a52d7054ea9aa59ccbac1/xxhash-3.4.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (194 kB)[2K     [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m194.8/194.8 kB[0m [31m7.1 MB/s[0m eta [36m0:00:00[0m[?25hCollecting multiprocess (from datasets)Downloading https://mirrors.cloud.tencent.com/pypi/packages/50/15/b56e50e8debaf439f44befec5b2af11db85f6e0f344c3113ae0be0593a91/multiprocess-0.70.16-py311-none-any.whl (143 kB)[2K     [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m143.5/143.5 kB[0m [31m944.8 kB/s[0m eta [36m0:00:00[0ma [36m0:00:01[0m[?25hRequirement already satisfied: fsspec<=2024.5.0,>=2023.1.0 in /opt/conda/lib/python3.11/site-packages (from fsspec[http]<=2024.5.0,>=2023.1.0->datasets) (2024.3.1)Collecting aiohttp (from datasets)Downloading https://mirrors.cloud.tencent.com/pypi/packages/24/99/e76e65ca811100b445d3c8af9764b27c5180ca11a15af694366424896647/aiohttp-3.9.5-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)[2K     [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m1.3/1.3 MB[0m [31m8.9 MB/s[0m eta [36m0:00:00[0m00:01[0m00:01[0m[?25hRequirement already satisfied: huggingface-hub>=0.21.2 in /opt/conda/lib/python3.11/site-packages (from datasets) (0.24.1)Requirement already satisfied: packaging in /opt/conda/lib/python3.11/site-packages (from datasets) (24.0)Requirement already satisfied: pyyaml>=5.1 in /opt/conda/lib/python3.11/site-packages (from datasets) (6.0.1)Collecting aiosignal>=1.1.2 (from aiohttp->datasets)Downloading https://mirrors.cloud.tencent.com/pypi/packages/76/ac/a7305707cb852b7e16ff80eaf5692309bde30e2b1100a1fcacdc8f731d97/aiosignal-1.3.1-py3-none-any.whl (7.6 kB)Requirement already satisfied: attrs>=17.3.0 in /opt/conda/lib/python3.11/site-packages (from aiohttp->datasets) (23.2.0)Collecting frozenlist>=1.1.1 (from aiohttp->datasets)Downloading https://mirrors.cloud.tencent.com/pypi/packages/b3/c9/0bc5ee7e1f5cc7358ab67da0b7dfe60fbd05c254cea5c6108e7d1ae28c63/frozenlist-1.4.1-cp311-cp311-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (272 kB)[2K     [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m272.3/272.3 kB[0m [31m9.3 MB/s[0m eta [36m0:00:00[0m[?25hCollecting multidict<7.0,>=4.5 (from aiohttp->datasets)Downloading https://mirrors.cloud.tencent.com/pypi/packages/52/ec/be54a3ad110f386d5bd7a9a42a4ff36b3cd723ebe597f41073a73ffa16b8/multidict-6.0.5-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (128 kB)[2K     [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m128.7/128.7 kB[0m [31m63.7 kB/s[0m eta [36m0:00:00[0ma [36m0:00:01[0m[?25hCollecting yarl<2.0,>=1.0 (from aiohttp->datasets)Downloading https://mirrors.cloud.tencent.com/pypi/packages/9f/ea/94ad7d8299df89844e666e4aa8a0e9b88e02416cd6a7dd97969e9eae5212/yarl-1.9.4-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (328 kB)[2K     [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m328.1/328.1 kB[0m [31m6.5 MB/s[0m eta [36m0:00:00[0ma [36m0:00:01[0m[?25hRequirement already satisfied: typing-extensions>=3.7.4.3 in /opt/conda/lib/python3.11/site-packages (from huggingface-hub>=0.21.2->datasets) (4.10.0)Requirement already satisfied: charset-normalizer<4,>=2 in /opt/conda/lib/python3.11/site-packages (from requests>=2.32.2->datasets) (3.3.2)Requirement already satisfied: idna<4,>=2.5 in /opt/conda/lib/python3.11/site-packages (from requests>=2.32.2->datasets) (3.6)Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/conda/lib/python3.11/site-packages (from requests>=2.32.2->datasets) (2.2.1)Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.11/site-packages (from requests>=2.32.2->datasets) (2024.7.4)Requirement already satisfied: python-dateutil>=2.8.2 in /opt/conda/lib/python3.11/site-packages (from pandas->datasets) (2.9.0)Requirement already satisfied: pytz>=2020.1 in /opt/conda/lib/python3.11/site-packages (from pandas->datasets) (2024.1)Requirement already satisfied: tzdata>=2022.7 in /opt/conda/lib/python3.11/site-packages (from pandas->datasets) (2024.1)Requirement already satisfied: six>=1.5 in /opt/conda/lib/python3.11/site-packages (from python-dateutil>=2.8.2->pandas->datasets) (1.16.0)Installing collected packages: xxhash, tqdm, requests, multiprocess, multidict, frozenlist, yarl, aiosignal, aiohttp, datasetsAttempting uninstall: tqdmFound existing installation: tqdm 4.66.2Uninstalling tqdm-4.66.2:Successfully uninstalled tqdm-4.66.2Attempting uninstall: requestsFound existing installation: requests 2.31.0Uninstalling requests-2.31.0:Successfully uninstalled requests-2.31.0Successfully installed aiohttp-3.9.5 aiosignal-1.3.1 datasets-2.20.0 frozenlist-1.4.1 multidict-6.0.5 multiprocess-0.70.16 requests-2.32.3 tqdm-4.66.4 xxhash-3.4.1 yarl-1.9.4Looking in indexes: https://mirrors.cloud.tencent.com/pypi/simpleCollecting einopsDownloading https://mirrors.cloud.tencent.com/pypi/packages/44/5a/f0b9ad6c0a9017e62d4735daaeb11ba3b6c009d69a26141b258cd37b5588/einops-0.8.0-py3-none-any.whl (43 kB)[2K     [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m43.2/43.2 kB[0m [31m1.9 MB/s[0m eta [36m0:00:00[0m[?25hInstalling collected packages: einopsSuccessfully installed einops-0.8.0

Task 2 利用 Intel Extension for Transformers 部署通义千问 Qwen-7B Chat，并根据 prompt 工程指定模型输出；完成模型的加载转换、部署。

请按照以下作业流程依次完成。

1. prompt 设计：利用特定分隔符完成指定格式的输出

任务要求：

1.1 给定一个段落让模型用一句话总结，给定段落如下：

《骆驼祥子》
祥子来自农村，是个破产的青年农民，勤劳、纯朴、善良，保留着农村哺育他、教养他的一切，却再也不愿意回农村去了。从农村来到城市的祥子，渴望以自己的诚实劳动买一辆属于自己的车。做个独立的劳动者是祥子的志愿、希望、甚至是宗教，凭着勤劳和坚忍，他用三年的时间省吃俭用，终于实现了理想，成为自食其力的上等车夫。但刚拉半年，车就在兵荒马乱中被逃兵掳走，祥子失去了洋车，只牵回三匹骆驼。祥子没有灰心，他依然倔强地从头开始，更加克己地拉车攒钱。可是，还没有等他再买上车，所有的积蓄又被侦探敲诈、洗劫一空，买车的梦想再次成泡影。
祥子又一次拉上自己的车是以与虎妞成就畸形的婚姻为代价的。可惜好景不长，因虎妞死于难产，他不得不卖掉人力车去料理丧事。至此，他的人生理想彻底破灭了。再加上他心爱的女人小福子的自杀，吹熄了他心中最后一朵希望的火花。连遭生活的打击，祥子开始丧失了对于生活的任何企求和信心，再也无法鼓起生活的勇气，不再像从前一样以拉车为自豪，他厌恶拉车，厌恶劳作。
被生活捉弄的祥子开始游戏生活，吃喝嫖赌。为了喝酒，祥子到处骗钱，堕落为“城市垃圾”。最后，靠给人干红白喜事做杂工维持生计。祥子由一个“体面的、要强的、好梦想的、利己的、个人的、健壮的、伟大的”底层劳动者沦为一个“堕落的、自私的、不幸的、社会病胎里的产儿，个人主义的末路鬼”

1.2 给出该段话的英文翻译

1.3 给出这段话中能反映情绪的关键词

1.4 将最终结果以 json 格式输出

# 导入 Intel Extension for Transformers 相关函数
from transformers import AutoTokenizer, TextStreamer
from intel_extension_for_transformers.transformers import AutoModelForCausalLM

# Your Answer：Prompt设计
prompt = '''根据给定的文本内容，完成以下任务：
1. 段落翻译为英文，注意词汇"底层劳动者"、"末路鬼"的翻译。
2. 给出段落的一句话总结。
3. 给出这段话的情绪关键词列表。文本内容：《骆驼祥子》祥子来自农村，是个破产的青年农民，勤劳、纯朴、善良，保留着农村哺育他、教养他的一切，却再也不愿意回农村去了。从农村来到城市的祥子，渴望以自己的诚实劳动买一辆属于自己的车。做个独立的劳动者是祥子的志愿、希望、甚至是宗教，凭着勤劳和坚忍，他用三年的时间省吃俭用，终于实现了理想，成为自食其力的上等车夫。但刚拉半年，车就在兵荒马乱中被逃兵掳走，祥子失去了洋车，只牵回三匹骆驼。祥子没有灰心，他依然倔强地从头开始，更加克己地拉车攒钱。可是，还没有等他再买上车，所有的积蓄又被侦探敲诈、洗劫一空，买车的梦想再次成泡影。祥子又一次拉上自己的车是以与虎妞成就畸形的婚姻为代价的。可惜好景不长，因虎妞死于难产，他不得不卖掉人力车去料理丧事。至此，他的人生理想彻底破灭了。再加上他心爱的女人小福子的自杀，吹熄了他心中最后一朵希望的火花。连遭生活的打击，祥子开始丧失了对于生活的任何企求和信心，再也无法鼓起生活的勇气，不再像从前一样以拉车为自豪，他厌恶拉车，厌恶劳作。被生活捉弄的祥子开始游戏生活，吃喝嫖赌。为了喝酒，祥子到处骗钱，堕落为“城市垃圾”。最后，靠给人干红白喜事做杂工维持生计。祥子由一个“体面的、要强的、好梦想的、利己的、个人的、健壮的、伟大的”底层劳动者沦为一个“堕落的、自私的、不幸的、社会病胎里的产儿，个人主义的末路鬼”。以json格式输出任务结果，格式规范，易于阅读和解析'''

# # Your Answer：Prompt设计
# prompt = '''
# 根据文本内容完成任务：
# 分别完成"英文翻译”、“一句话总结”、“情绪关键词”，结果的输出格式为json。
# {"英文翻译":,"一句话总结":,"情绪关键词":}
# 文本内容：《骆驼祥子》
# 祥子来自农村，是个破产的青年农民，勤劳、纯朴、善良，保留着农村哺育他、教养他的一切，却再也不愿意回农村去了。从农村来到城市的祥子，渴望以自己的诚实劳动买一辆属于自己的车。做个独立的劳动者是祥子的志愿、希望、甚至是宗教，凭着勤劳和坚忍，他用三年的时间省吃俭用，终于实现了理想，成为自食其力的上等车夫。但刚拉半年，车就在兵荒马乱中被逃兵掳走，祥子失去了洋车，只牵回三匹骆驼。祥子没有灰心，他依然倔强地从头开始，更加克己地拉车攒钱。可是，还没有等他再买上车，所有的积蓄又被侦探敲诈、洗劫一空，买车的梦想再次成泡影。
# 祥子又一次拉上自己的车是以与虎妞成就畸形的婚姻为代价的。可惜好景不长，因虎妞死于难产，他不得不卖掉人力车去料理丧事。至此，他的人生理想彻底破灭了。再加上他心爱的女人小福子的自杀，吹熄了他心中最后一朵希望的火花。连遭生活的打击，祥子开始丧失了对于生活的任何企求和信心，再也无法鼓起生活的勇气，不再像从前一样以拉车为自豪，他厌恶拉车，厌恶劳作。
# 被生活捉弄的祥子开始游戏生活，吃喝嫖赌。为了喝酒，祥子到处骗钱，堕落为“城市垃圾”。最后，靠给人干红白喜事做杂工维持生计。祥子由一个“体面的、要强的、好梦想的、利己的、个人的、健壮的、伟大的”底层劳动者沦为一个“堕落的、自私的、不幸的、社会病胎里的产儿，个人主义的末路鬼”
# '''

# # Your Answer：Prompt设计
# prompt = '''
# 根据文本内容完成任务：
# 1. 给出该段话的英文翻译。
# 2. 给出文本的一句话总结。
# 3. 给出这段话能反映情绪的关键词。
# 结果以json的格式输出，分别对应“文本翻译”、“一句话总结”、“情绪关键词”。
# 文本内容：《骆驼祥子》
# 祥子来自农村，是个破产的青年农民，勤劳、纯朴、善良，保留着农村哺育他、教养他的一切，却再也不愿意回农村去了。从农村来到城市的祥子，渴望以自己的诚实劳动买一辆属于自己的车。做个独立的劳动者是祥子的志愿、希望、甚至是宗教，凭着勤劳和坚忍，他用三年的时间省吃俭用，终于实现了理想，成为自食其力的上等车夫。但刚拉半年，车就在兵荒马乱中被逃兵掳走，祥子失去了洋车，只牵回三匹骆驼。祥子没有灰心，他依然倔强地从头开始，更加克己地拉车攒钱。可是，还没有等他再买上车，所有的积蓄又被侦探敲诈、洗劫一空，买车的梦想再次成泡影。
# 祥子又一次拉上自己的车是以与虎妞成就畸形的婚姻为代价的。可惜好景不长，因虎妞死于难产，他不得不卖掉人力车去料理丧事。至此，他的人生理想彻底破灭了。再加上他心爱的女人小福子的自杀，吹熄了他心中最后一朵希望的火花。连遭生活的打击，祥子开始丧失了对于生活的任何企求和信心，再也无法鼓起生活的勇气，不再像从前一样以拉车为自豪，他厌恶拉车，厌恶劳作。
# 被生活捉弄的祥子开始游戏生活，吃喝嫖赌。为了喝酒，祥子到处骗钱，堕落为“城市垃圾”。最后，靠给人干红白喜事做杂工维持生计。祥子由一个“体面的、要强的、好梦想的、利己的、个人的、健壮的、伟大的”底层劳动者沦为一个“堕落的、自私的、不幸的、社会病胎里的产儿，个人主义的末路鬼”
# '''

# prompt = '''1给定一个段落让模型用一句话总结，给定段落如下：
# 《骆驼祥子》
# 祥子来自农村，是个破产的青年农民，勤劳、纯朴、善良，保留着农村哺育他、教养他的一切，却再也不愿意回农村去了。从农村来到城市的祥子，渴望以自己的诚实劳动买一辆属于自己的车。做个独立的劳动者是祥子的志愿、希望、甚至是宗教，凭着勤劳和坚忍，他用三年的时间省吃俭用，终于实现了理想，成为自食其力的上等车夫。但刚拉半年，车就在兵荒马乱中被逃兵掳走，祥子失去了洋车，只牵回三匹骆驼。祥子没有灰心，他依然倔强地从头开始，更加克己地拉车攒钱。可是，还没有等他再买上车，所有的积蓄又被侦探敲诈、洗劫一空，买车的梦想再次成泡影。
# 祥子又一次拉上自己的车是以与虎妞成就畸形的婚姻为代价的。可惜好景不长，因虎妞死于难产，他不得不卖掉人力车去料理丧事。至此，他的人生理想彻底破灭了。再加上他心爱的女人小福子的自杀，吹熄了他心中最后一朵希望的火花。连遭生活的打击，祥子开始丧失了对于生活的任何企求和信心，再也无法鼓起生活的勇气，不再像从前一样以拉车为自豪，他厌恶拉车，厌恶劳作。
# 被生活捉弄的祥子开始游戏生活，吃喝嫖赌。为了喝酒，祥子到处骗钱，堕落为“城市垃圾”。最后，靠给人干红白喜事做杂工维持生计。祥子由一个“体面的、要强的、好梦想的、利己的、个人的、健壮的、伟大的”底层劳动者沦为一个“堕落的、自私的、不幸的、社会病胎里的产儿，个人主义的末路鬼”
# 2 给出该段话的英文翻译
# 3 给出这段话中能反映情绪的关键词
# 将最终结果以 json 格式输出'''

2. 部署模型：在 /temp 目录下完成 Qwen-7B Chat 的加载、部署

任务要求：

2.1 通过在数据集中外挂 Qwen-7B Chat 或者通过魔搭下载 Qwen-7B Chat 的方式完成模型加载

2.2 通过 AutoTokenizer 完成 tokenizer 的加载

2.3 通过 Intel Extension for Transformers 的 AutoModelForCausalLM 工具快速完成模型部署

# Your Answer：模型加载和部署
model_dir = '/home/mw/input/qwen7bchat7438'
tokenizer = AutoTokenizer.from_pretrained(model_dir, trust_remote_code=True)
inputs = tokenizer(prompt, return_tensors="pt").input_ids
streamer = TextStreamer(tokenizer)
model = AutoModelForCausalLM.from_pretrained(model_dir, load_in_4bit=True, trust_remote_code=True)
# model = AutoModelForCausalLM.from_pretrained(model_dir, trust_remote_code=True) # 尝试直接加载FP32模型

2024-07-24 08:12:53 [INFO] cpu device is used.
2024-07-24 08:12:53 [INFO] Applying Weight Only Quantization.
2024-07-24 08:12:53 [INFO] Quantize model by Neural Speed with RTN Algorithm.runtime_outs/ne_qwen_q_nf4_bestla_cfp32_g32.bin existed, will use cache file. Otherwise please remove the file

​

3. 输出模型：输入作业 prompt 得到模型输出

outputs = model.generate(inputs, streamer=streamer, max_new_tokens=550)
# outputs

load_ne_hparams  0.hparams.n_vocab = 151936                        
load_ne_hparams  1.hparams.n_embd = 4096                          
load_ne_hparams  2.hparams.n_mult = 22016                         
load_ne_hparams  3.hparams.n_head = 32                            
load_ne_hparams  4.hparams.n_head_kv = 0                             
load_ne_hparams  5.hparams.n_layer = 32                            
load_ne_hparams  6.hparams.n_rot = 128                           
load_ne_hparams  7.hparams.ftype = 0                             
load_ne_hparams  8.hparams.max_seq_len = 8192                          
load_ne_hparams  9.hparams.alibi_bias_max = 0.000                         
load_ne_hparams  10.hparams.clip_qkv = 0.000                         
load_ne_hparams  11.hparams.par_res = 0                             
load_ne_hparams  12.hparams.word_embed_proj_dim = 0                             
load_ne_hparams  13.hparams.do_layer_norm_before = 0                             
load_ne_hparams  14.hparams.multi_query_group_num = 0                             
load_ne_hparams  15.hparams.ffn_hidden_size = 11008                         
load_ne_hparams  16.hparams.inner_hidden_size = 0                             
load_ne_hparams  17.hparams.n_experts = 0                             
load_ne_hparams  18.hparams.n_experts_used = 0                             
load_ne_hparams  19.hparams.n_embd_head_k = 0                             
load_ne_hparams  20.hparams.norm_eps = 0.000001                      
load_ne_hparams  21.hparams.freq_base = 10000.000                     
load_ne_hparams  22.hparams.freq_scale = 1.000                         
load_ne_hparams  23.hparams.rope_scaling_factor = 0.000                         
load_ne_hparams  24.hparams.original_max_position_embeddings = 0                             
load_ne_hparams  25.hparams.use_yarn = 0                             
load_ne_vocab    26.vocab.bos_token_id = 151643                        
load_ne_vocab    27.vocab.eos_token_id = 151643                        
load_ne_vocab    28.vocab.pad_token_id = -1                            
load_ne_vocab    29.vocab.sep_token_id = -1                            
beam_size: 1, do_sample: 0, top_k: 40, top_p: 0.950, continuous_batching: 0, max_request_num: 1, early_stopping: 0, scratch_size_ratio: 1.000
Loading the bin file with NE format...
根据给定的文本内容，完成以下任务：
1. 段落翻译为英文，注意词汇"底层劳动者"、"末路鬼"的翻译。
2. 给出段落的一句话总结。
3. 给出这段话的情绪关键词列表。文本内容：《骆驼祥子》祥子来自农村，是个破产的青年农民，勤劳、纯朴、善良，保留着农村哺育他、教养他的一切，却再也不愿意回农村去了。从农村来到城市的祥子，渴望以自己的诚实劳动买一辆属于自己的车。做个独立的劳动者是祥子的志愿、希望、甚至是宗教，凭着勤劳和坚忍，他用三年的时间省吃俭用，终于实现了理想，成为自食其力的上等车夫。但刚拉半年，车就在兵荒马乱中被逃兵掳走，祥子失去了洋车，只牵回三匹骆驼。祥子没有灰心，他依然倔强地从头开始，更加克己地拉车攒钱。可是，还没有等他再买上车，所有的积蓄又被侦探敲诈、洗劫一空，买车的梦想再次成泡影。祥子又一次拉上自己的车是以与虎妞成就畸形的婚姻为代价的。可惜好景不长，因虎妞死于难产，他不得不卖掉人力车去料理丧事。至此，他的人生理想彻底破灭了。再加上他心爱的女人小福子的自杀，吹熄了他心中最后一朵希望的火花。连遭生活的打击，祥子开始丧失了对于生活的任何企求和信心，再也无法鼓起生活的勇气，不再像从前一样以拉车为自豪，他厌恶拉车，厌恶劳作。被生活捉弄的祥子开始游戏生活，吃喝嫖赌。为了喝酒，祥子到处骗钱，堕落为“城市垃圾”。最后，靠给人干红白喜事做杂工维持生计。祥子由一个“体面的、要强的、好梦想的、利己的、个人的、健壮的、伟大的”底层劳动者沦为一个“堕落的、自私的、不幸的、社会病胎里的产儿，个人主义的末路鬼”。以json格式输出任务结果，格式规范，易于阅读和解析model.cpp: loading model from runtime_outs/ne_qwen_q_nf4_bestla_cfp32_g32.bin
init: n_vocab    = 151936
init: n_embd     = 4096
init: n_mult     = 22016
init: n_head     = 32
init: n_head_kv  = 0
init: n_layer    = 32
init: n_rot      = 128
init: ftype      = 0
init: max_seq_len= 8192
init: n_ff       = 11008
init: n_parts    = 1
load: ctx size   = 4581.78 MB
load: scratch0   = 4096.00 MB
load: scratch1   = 2048.00 MB
load: scratch2   = 4096.00 MB
load: mem required  = 14821.78 MB (+ memory per state)
......................................................................................
model_init_from_file: support_bestla_kv = 0
model_init_from_file: kv self size =  256.00 MB。​```json
{"段落翻译": "From the countryside to the city, Wang Laozi comes from a rural background, a bankrupt young farmer, hardworking, pure, kind, retaining everything that fed and educated him in the countryside, but no longer willing to return to the countryside. From the countryside to the city, Wang Laozi, who desires to be an independent laborer, hopes, even religion, relies on hard work and perseverance to save money in three years, finally realizing his ideal and becoming an independent laborer. But after half a year, the car was taken away by soldiers in chaos. Wang Laozi lost his foreign car, only taking back three camels. Wang Laozi did not lose heart, he still stubbornly started again, working harder to save money. But before he could buy another car, all his savings were stolen by detectives. The dream of buying a car again became a mirage. Wang Laozi once again bought a car by marrying a deformed woman, Tiger. Unfortunately, it didn't last long, because Tiger died in childbirth, he had to sell his labor car to handle the funeral. At this point, his life ideal was completely shattered. In addition to his beloved woman, Fuyu's suicide, he extinguished the last spark of hope in his heart. After being caught in the twists and turns of life, Wang Laozi began to disdain life. He played with life, eating, drinking, gambling. To drink, Wang Laozi went around to beg for money, degenerating into "urban garbage". Finally, he supported himself by doing odd jobs for weddings and funerals. Wang Laozi, a "respectable, strong, good dreamer, selfish, individualistic, healthy, great" lower-level laborer, became a "degenerate, selfish, unfortunate, a product of the diseased womb of society, the end of individualism.""一句话总结": "祥子由一个体面的、要强的、好梦想的、利己的、个人的、健壮的、伟大的底层劳动者沦为一个堕落的、自私的、不幸的、社会病胎里的产儿，个人主义的末路鬼。""情绪关键词列表": ["绝望", "堕落", "痛苦", "失望", "愤怒", "无奈", "悲哀"]
}
​```<|endoftext|>

# 解码输出
# output_str = tokenizer.decode(outputs[0], skip_special_tokens=True)
# print("Generated text:", output_str)

提交指引：

完成作业后，请注意保存进度，并为项目生成最新版本，然后打开 提交页面 - 项目作业 提交这个项目。

活动导师会尽快人工评审你的项目，评审通过后，你将晋级到 通关！ 阶段，成功获得 3 个夏令营学分。